ensuring a directory exists and is opened

[schveiguy]

I'm having trouble ensuring that a directory is present and opened.  Here is the code I have:

Code: [Select]

static int openDirectory()
{
    int result = UTFAT_SUCCESS;
    if(sdDirectory == NULL)
    {
        sdDirectory = utAllocateDirectory(DISK_D, 0);
    }
    result = utOpenDirectory("/CONF", sdDirectory);
    if(result == UTFAT_FILE_NOT_FOUND)
    {
        // need to create the directory
        result = utMakeDirectory("/CONF", sdDirectory);
        if(result == UTFAT_SUCCESS)
        {
            result = utOpenDirectory("/CONF", sdDirectory);
        }
    }
    return result;
}
The first time this function is called, the directory D:\CONF is created and opened
Upon the second time calling this, the directory D:\CONF\CONF is created, but D:\CONF is opened.

I tried adding a statement before opening the directory to clear the VALID bit, but what that results in is the directory D:\CONF being created, but D:\ being opened.

I'm unsure how the directory argument to utOpenDirectory is used, I thought it was just a place to put the opened directory, but it appears to play a role in the relation of the directory to open.  Plus, makeDirectory doesn't have any docs, so I'm not sure how it's used there either.

What I want is for this function to work even if someone has removed the directory between calls.

-Steve

[mark]

Hi Steve

You are using a directory object without any path. This means that you can move into directories (downwards) but not upwards. You can however always go back to the root directory.

What is happening in this case is that the first time you call the utOpenDirectory() you are located at the root and this directory can't be opened since it presently doesn't exist. Now you create it and you are still situated at the root with a directory called "CONF" in it so you open this directory and thus are now at D:/CONF.

If you again enter the function and try to open the directory "CONF" it is not finding it in your directory and thus creates it again - then you move to D:/CONF/CONF, which repeats.

You may have been expecting that "/CONF" would mean that it always checks referenced to the root and not at the present location. You may also be correct in assuming this but I have a feeling that the present release version doesn't do this correctly or else it is a side-effect of the fact that you don't have a path (don't see why though really).

You can do the following:
- before opening the first directory you can invalidate the directory location using
sdDirectory->usDirectoryFlags &= ~UTDIR_VALID;
which will set you back to root. Then the check will always be for the directory "CONF" in the root directory and so it will always find it once it has once been created which will operate the way that you are expecting it to.

I will look into the "/" use in the meantime. I know that it is used by some tester with the latest development code but I think that the doc is slightly ahead of the last release in this respect. I would like to release a new utFAT by itself but there are a couple of incompatibilities with the HTTP and FTP server code so have been putting it off until full releases are made. This should be resolved quite shortly though with a set of releases in preparation.

Tell me if this workaround is OK for you.

Regards

Mark

[schveiguy]

You can do the following:
- before opening the first directory you can invalidate the directory location using
sdDirectory->usDirectoryFlags &= ~UTDIR_VALID;
which will set you back to root. Then the check will always be for the directory "CONF" in the root directory and so it will always find it once it has once been created which will operate the way that you are expecting it to.

In fact, I did try this (I noted it before, but didn't spell it out exactly).  Here is my current code.  But it results in files being created in the root directory.  I.e. it appears that on the second openDirectory the system opens D:\ instead of D:\CONF.

Code: [Select]

static int openDirectory()
{
    int result = UTFAT_SUCCESS;
    if(sdDirectory == NULL)
    {
        sdDirectory = utAllocateDirectory(DISK_D, 0);
    }
    sdDirectory->ucDirectoryFlags &= (~UTDIR_VALID); // reset so we don't open a relative directory
    result = utOpenDirectory("/CONF", sdDirectory);
    if(result == UTFAT_FILE_NOT_FOUND)
    {
        // need to create the directory
        result = utMakeDirectory("/CONF", sdDirectory);
        if(result == UTFAT_SUCCESS)
        {
            sdDirectory->ucDirectoryFlags &= (~UTDIR_VALID);
            result = utOpenDirectory("/CONF", sdDirectory); // this seems to open D:
        }
    }
    return result;
}


[mark]

Hi Steve

I have attached a new version of utFATV1.6. This should be compatible with all V1.4 based projects and included the following files:
- mass_storage.c (utFAT1.6)
- mass_storage.h (utFAT1.6)
- SDcard_sim.c (SD card simulator)
- ftp.c and http.c (due to slightly adjustment in interface)
- debug.c (also due to slight adjustment in interface)

I think that it will be easier if the most up to date version is used since it includes all patches and latest features.

In this particular case I understand the subject to be the difference between using mkdir CONF and mkdir /CONF. That is, whether the directory to be created is referenced to the actual directory location or to the root directory (D:/).

The following sequence can be tested with this version:
1) format (or re-format) the card so that it is empty
2) command mkdir CONF
3) command cd CONF - we are now in the directory D:/CONF
4) command mkdir /CONF - this must fail because it is commanding the creation of the directory D:/CONF which already exists
5) command mkdir CONF - this must be successful since it creates D:/CONF/CONF

When I tested your code I didn't actually see what it was doing wrong because it did essentially the same (once created a second creation was no longer possible as ling as the / was used to reference the root directory).

This version also allows full referenced file operation according to the user's guide. As long as a directory path string is used thing like mkdir ../../test/test1 are possible. Without using directory paths (as in the case when the directory object is allocated with utAllocateDirectory(DISK_D, 0); upward referencing can not be used but still root referencing can, eg. mkdir /test/test1 assuming that test is in the root directory, whereby the present directory location is not important and will also not be changed.

In fact to do the same thing the following can be used. It is not necessarily the best code but it should show the principle:

    utMakeDirectory("/CONF", sdDirectory);
    utOpenDirectory("/CONF", sdDirectory);

Assuming that this is called a number of times with an empty card it will first crate the directory D:/CONF. Then it will open it and move to D:/CONF.
When called a second time the create will fail and the same directory location will be opened - this will not fail but simply go to where it already is. This can be repeated indefinitely without any consequences.

However the following is a very different case:
    utMakeDirectory("CONF", sdDirectory);
    utOpenDirectory("CONF", sdDirectory);
It will first create D:/CONF and move to there. When called a second time it will create D:/CONF/CONF and move there. The next time it will create D:/CONF/CONF/CONF and move there.
Each time it will create a new directory at the next level and move there and is it would get infinitely deep directories if the call were continued.

I have just spend some time checking behavior in various cases and am pleased with the results as all tests that I did operated as expected. However do note that this is the latest development code (utFAT1.6 as reference) and so latest changes can always have side effects that will only be noticed with further exercising and time.

See how you get on with this - it will be better when all can work with the most up to date version and then I will try to extend the documentation (i had forgotten that there are still various function that are not fully documented..)

regards

Mark

[schveiguy]

I installed the new SDCard_Sim.c, the new mass_storage.[ch], and did a global search/replace of ucDirectoryFlags -> usDirectoryFlags.  I didn't want to update the other files, because I've modified them.

Now, the directory is created correctly, but my files aren't written *at all*

I'm going to do some more investigation, and see if I can narrow down my code to a small test case that you can use to reproduce, or at least see if I've done something wrong.

-Steve

[update]
OK, what I've found is, when it opens the directory, it returns UTFAT_PATH_IS_ROOT_REF instead of UTFAT_SUCCESS.  Is this valid?  My code was looking for UTFAT_SUCCESS, so it skipped trying to write any data.

There seems to be no comment to describe what this means, can you elaborate?

[schveiguy]

More updates, I'm assuming that return value is success, and it's still writing files opened with that directory to D:\ not D:\CONF.  Maybe I'm misunderstanding how this works. 

Here is some code that duplicates the problem:

Code: [Select]

static UTDIRECTORY *sdDirectory = NULL;

static int openDirectory()
{
    int result = UTFAT_SUCCESS;
    if(sdDirectory == NULL)
    {
        sdDirectory = utAllocateDirectory(DISK_D, 0);
    }
    result = utOpenDirectory("/CONF", sdDirectory);
    if(result == UTFAT_FILE_NOT_FOUND)
    {
        // need to create the directory
        result = utMakeDirectory("/CONF", sdDirectory);
        if(result == UTFAT_SUCCESS)
        {
            result = utOpenDirectory("/CONF", sdDirectory);
        }
    }
    if(result == UTFAT_PATH_IS_ROOT_REF)
        // not sure why we have to do this...
        result = UTFAT_SUCCESS;
    return result;
}

int conf_store()
{
    UTFILE newFile;
    int result = 0;
    // first, we try to store to the SD card file
    if(!openDirectory())
    {
        newFile.ptr_utDirObject = sdDirectory;
        // write the new file
        if(utOpenFile("TEST.CNF", &newFile, UTFAT_OPEN_FOR_WRITE | UTFAT_TRUNCATE | UTFAT_CREATE) == UTFAT_PATH_IS_FILE)
        {
        if((result = utWriteFile(&newFile, "hello\r\n", 7)) != UTFAT_SUCCESS)
            return result;
        }
    }
    return 0;
}

just call conf_store, and I would expect to have a D:\CONF directory with a file TEST.CNF in it.  Instead, TEST.CNF gets put into D:\

-Steve

[mark]

Hi Steve

UTFAT_PATH_IS_ROOT_REF is used as return value to distinguish this case by use when navigating through directories. It is not (usually) relevant to the user. Since error cases are negative, the best way to test for errors is thus:
if (result < UTFAT_SUCCESS)

I tested your code and confirm that it doesn't move to the expected directory as you write. Before explaining why, it is best to know a couple of details about the directory object:
This has three objects which contain information about directories belonging to the directory object itself:
    DISK_LOCATION              root_disk_location;                       // reference to the root directory
    DISK_LOCATION              private_disk_location;                    // details of where the directory is physically located on the disk
    DISK_LOCATION              public_disk_location;                     // working information when searching for and manipulating directory contents


The root_disk_location may be the root directory of the disk but it may also be the root directory of its use (restricted to certain sub-directory).
The private_disk_location is supposed to be the location that the user is at (it can be anywhere below its root).
The public_disk_location is a temporary entry which is used when operations are performed (its value is a sort of working area.

This means that one may expect the private_disk_location changes when a directory is opened. This also happens when a directory is opened when the object is not yet valid. When using utChangeDirectory() this is also correctly performed, but utOpenDirectory() is also used to locate sub-directories without actually moving there so that remote operations can be performed.

This is maybe not logical and it would probably be best if the utOpenDirectory() as used by the user were to move as utChangeDirectory() does. I suppose that utChangeDirectory() is more or less the utOpenDirectory() as one would expect the latter to work.

It is however rather sensitive to change the utOpenDirectory() so that it always moves because it then causes other uses to change in behavior. This means that I am going to have to study this more before making a decision as to its best solution.

A workaround for you is to use:
result = utChangeDirectory("/CONF", sdDirectory);
instead of
result = utOpenDirectory("/CONF", sdDirectory);
[this also returns UTFAT_SUCCESS]. Only use the open to open the first directory when the object is not yet valid.

Note that the navigation as performed by the DOS like interface and also FTP uses utOpenDirectory() to see whether the location exists and this always moves there when the directory object is used for the first time. Then movement is then controlled by utChangeDirectory(). This does work well but now I am wondering whether this is really the most logical method or is the functionality of utChangeDirectory() more one would expect from utOpenDirectory()?

Regards

Mark