Files saved on SD card dont show on PC

[neil]

Hi Mark,
  I am using the latest version of the SD source, v2.01 with changes to some SD cards where they didn't format.    These cards are formatted as FAT32, and the files are created and read ok when connected to the processor. When I remove the card and place in the PC , it says the card is empty. The properties shows that 1.5MB has been used, but now files are displayed.

Regards
Neil

[mark]

Hi Neil

I haven't heard of such an effect. Is this reproducible with all cards or just a few?

What you could do is use the HDD Raw Copy Tool "HDDRawCopy V1.1" to make an image of the card and send it to me. I can then copy the exact image to a card (including one that sent me that was causing initialisation problems) so that I can see it happening and possible explain it.

Regards

Mark

[neil]

Hi Mark
  The 2 types of cards we have are sandisk , and transcend  cards. And it happens on both types. I have attached the file.

Regards
Neil

[neil]

Hi Mark
  Any luck with the file?

Regards
Neil

[mark]

Neil

I have used this to create an image on a 4GByte SanDisk card.

In the PC I see it as a disk called "QWIKDISK" with no data on it.

When I insert it into a Kinetis board (SDHC) I see this as info:

Disk QWIKDISK (3965190144 bytes) SDHC FAT32
Bytes per sector: 512
Cluster size: 4096
Directory base: 0x00000002
FAT start: 0x0000005f
FAT size: 0x00000ef3
Number of FATs: 2
LBA: 0x00001e45
Total clusters: 0x00077737
Info sect: 0x00000040
Free clusters: 0x00076da8
Next free: 0x00000991
CSD: 0x40 0x0e 0x00 0x32 0x5b 0x59 0x00 0x00 0x1d 0x8a 0x7f 0x80 0x0a 0x40 0x40 0x00

The "dir" commnd shows it as empty.
The "dird" command shows that there was a file deleted called "ReadyBoostPerfTest.tmp"

[D]---A 24.08.2015  11:44          8388608 ReadyBoostPerfTest.tmp
1 files with 8388608 bytes
0 directories, 1994031104 bytes free


When I then analyse this file with the "infod" command I get:

D:\>infod ReadyBoostPerfTest.tmp
File: ReadyBoostPerfTest.tmp is LFN
Starting at entry 0x04 in sector 0x00001e45 (cluster 0x00000002)
Deleted LFN
Data = 0xe5 0x66 0x00 0x54 0x00 0x65 0x00 0x73 0x00 0x74 0x00 0x0f 0x00 0x49 0x2e 0x00 0x74 0x00 0x6d 0x00 0x70 0x00 0x00 0x00 0xff 0xff 0x00 0x00 0xff 0xff 0xff 0xff
Deleted LFN
Data = 0xe5 0x52 0x00 0x65 0x00 0x61 0x00 0x64 0x00 0x79 0x00 0x0f 0x00 0x49 0x42 0x00 0x6f 0x00 0x6f 0x00 0x73 0x00 0x74 0x00 0x50 0x00 0x00 0x00 0x65 0x00 0x72 0x00
Deleted LFN
End
SFN File located at entry 0x06 in sector 0x00001e45 (cluster 0x00000002)
Data = 0xe5 0x45 0x41 0x44 0x59 0x42 0x7e 0x31 0x54 0x4d 0x50 0x20 0x00 0x49 0x87 0x5d 0x18 0x47 0x18 0x47 0x00 0x00 0x89 0x5d 0x18 0x47 0x91 0x01 0x00 0x00 0x80 0x00
SFN name (archive) = .EADYB~1TMP Alias CS = 0x49
File length = 8388608 starting in sector 0x00002abd (cluster 0x00000191) FAT sector 0x00000062 offset 0x11

Finally, I check the sector content (0x1e45) where the deleted file object is located:

sect 1e45
Reading sector 0x00001e45
 0x57205141 0x2e006265 0x0872452e 0x60000000 0x44eb44eb 0x60000000 0x000044eb 0x00000000 0x49002042 0x66006e00 0x0f006f00 0x00727200 0x0061006d 0x00690074 0x0000006f 0x0000006e
 0x79005301 0x74007300 0x0f006500 0x006d7200 0x00560020 0x006c006f 0x00000075 0x0065006d 0x54535953 0x317e4d45 0x16202020 0x5d6f6800 0x47184718 0x5d700000 0x018f4718 0x00000000
 0x540066e5 0x73006500 0x0f007400 0x002e4900 0x006d0074 0x00000070 0x0000ffff 0xffffffff 0x650052e5 0x64006100 0x0f007900 0x00424900 0x006f006f 0x00740073 0x00000050 0x00720065
 0x444145e5 0x317e4259 0x20504d54 0x5d874900 0x47184718 0x5d890000 0x01914718 0x00800000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000

Here I see the same data content and no other file objects (following are only 0x00000000).

Therefore, it looks like there is really no data on the card (once the original single file was deleted).

Could you do the same to see whether there is a difference? What do your see on the card? If you find files on it can you use the "infof" to see its location on the disk so that I can check whether it is on it somewhere?

Regards

Mark

[neil]

Hi Mark,
   Hmmmm..  I think I must have copied the wrong card. When in operation, our board saves the data , and various setup for the board on the SD .  The board then sends the data that was saved to our server, so we know the data is saved. I took out the card and simply done opened it up on Windows. One of our customers done the same with their board. We have quite a few SD cards about, so it could have well been the wrong card. Is it okay if I (I will make sure this time it is the correct card) send another file?  Im on vacation at the moment, so will do it when I get back .

Best Regards
Neil

[mark]

Neil

Resend an image when you are back so that I can compare.

Regards

Mark

[neil]

Thanks Mark

Regards
Neil

[neil]

Hi Mark,
   I have ran my application now for several hours, and able to collect the data on the SD card, I can download the data at any point (I connect through tcp and download the data), the application opens the files and sends the data. I still cant see the data on a PC, when I run the test utasker  application and do a dir I get no files , and when I look at the info I get the following below.

If I restart my application , I can see the files (I have debug code) being opened during initialization of  the system, and when it downloads the data, I can see the files being opened, and data sent (which is correct info).

I have removed the sd card, and copied it now (attached), this time i know its teh same card

Following from dir and info:
Directory D:\

0 files with 0 bytes
0 directories, 1973481472 bytes free
D:\>info

SD-card  (1977614336 bytes)
Bytes per sector: 512
Cluster size: 4096
Directory base: 0x00000002
FAT start: 0x0000005f
FAT size: 0x00000eb9
Number of FATs: 2
LBA: 0x00001dd1
Total clusters: 0x00075a45
Info sect: 0x00000040
Free clusters: 0x00075a0f
Next free: 0x00000038
CSD: 0x00 0x26 0x00 0x32 0x5f 0x5a 0x83 0xae 0xfe 0xfb 0xcf 0xff 0x92 0x80 0x40
0xdf
D:\>

Regards
Neil

[mark]

Neil

With this image I can't see any content on the disk.

What I do see is that there is FAT content:

D:\>sect 5f
Reading sector 0x0000005f
 0x0ffffff8 0xffffffff 0x00000003 0x00000005 0x0fffffff 0x00000007 0x00000036 0x00000009 0x0fffffff 0x0000000b 0x0000000d 0x0fffffff 0x00000035 0x0000000f 0x0fffffff 0x00000011
 0x0fffffff 0x00000013 0x0fffffff 0x00000015 0x0fffffff 0x00000017 0x0fffffff 0x00000019 0x0fffffff 0x0000001b 0x0fffffff 0x0000001d 0x0fffffff 0x0000001f 0x0fffffff 0x00000021
 0x0fffffff 0x00000023 0x0fffffff 0x00000025 0x0fffffff 0x00000027 0x0fffffff 0x00000029 0x0fffffff 0x0000002b 0x0fffffff 0x0000002d 0x0fffffff 0x0000002f 0x0fffffff 0x00000031
 0x0fffffff 0x00000033 0x0fffffff 0x0fffffff 0x0fffffff 0x0fffffff 0x0fffffff 0x00000038 0x00000039 0x0000003a 0x0000003b 0x0000003c 0x0000003d 0x0000003e 0x0000003f 0x00000040
 0x00000041 0x00000042 0x00000043 0x00000044 0x00000045 0x00000046 0x00000047 0x00000048 0x00000049 0x0000004a 0x0000004b 0x0000004c 0x0000004d 0x0000004e 0x0000004f 0x00000050
 0x00000051 0x00000052 0x00000053 0x0fffffff 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
D:\>

This shows that there are some cluster chains (still) in use. For example, the root directories cluster chain starts in sector 1dd1 (Cluster 2 / 8 sectors per cluster), followed by 0x1dd9 (cluster 3), followed by 0x1de9 (cluster 5), followed by 0x1df9 (cluster 7), followed by 0x1e09 (cluster 9), followed by 0x1e19 (cluster 0xb) - end

What I notice is that each of these clusters has a single file in it (as if each were an independent sub-directory) with names like FLUSH, 2015, INDEX, CALIB etc., each marked as files.

There is also data content that can be seen in some clusters (possibly OK) but the strangest thing is that the root directory looks to be made up of a number of clusters, each with a single file name in each one. It is expected that the root (and any other directory) is made up of entries that are all alongside each other - each cluster can contain 128 such file objects (they are all short file names) and not just one per cluster (!)
The result is that the directory listing scans the cluster until it finds an empty entry signalling the end (which happens immediately since there is such a lot of space).

At the moment it is surprising that your firmware actually sees the content - it is clear that Windows also won't find anything, as the firmware that I am running doesn't. If I understnd correctly this is also the case when you take a 'standard' project build and only happens with your build (?)

At a first educated guess it looks like your code is not using 32 bytes as a directory entry size but instead using the cluster size (4k). File object increments look to be stepping by this amount. This supposedly works when written and read by code doing this but will not be understood by anything else.

I would: check with the debugger any code that is using a pointer to DIR_ENTRY_STRUCTURE_FAT32 and executing something like
ptrDirectoryEntry += ptr_disk_location->ucDirectoryEntry;        // move to the present entry
Each ptrDirectoryEntry++ should be a 32 byte increment, whereby it looks like your compiled code may be incrementing in 4k steps instead.
Has something happened to DIR_ENTRY_STRUCTURE_FAT32 that explains this or is the compiler going crazy??

Hope you find something quickly based on this data.

Regards

Mark

[neil]

Hi Mark
  Thanks for the reply. I will take a look and see if its this causing the issue. You are correct with the files on the card,  FLUSH, 2015, INDEX, CALIB . There are all files names in the root directory , and 2015 is a folder where data is stored.

Regards
Neil

[neil]

Hi Mark
  What routines use this? Is the size dictated during the format procedure?  I remember it used to work okay before the last SD latest version. I simply overwrote the .h and .c mass storage files. Was there anything else needed?


regards
Neil

[mark]

Hi Neil

Exchanging the utFAT files in its folder is all that is needed at an update.

I suggest doing the following as a test: reformat the card and the create a directory, monitoring the location of the directory object. This is how it should work, using the command line:

  Disk interface
===================
up           go to main menu
info         utFAT/card info
dir          [path] show directory content
dird         [path] show deleted directory content
dirh         [path] show hidden content
infof        [path] show file info
infod        [path] show deleted info
cd           [path] change dir. (.. for up)
file         [path] new empty file
write        [path] test write to file
mkdir        new empty dir
rename       [from] [to] rename
trunc        truncate to [length] [path]
print        [path] print file content
del          [path] delete file or dir.
undel        undelete [name]
format       [-16/12] [label] format (unformatted) disk
fformat      [-16/12] [label] full format (unformatted) disk
re-format    [-16/12] [label] reformat disk!!!!!
re-fformat   [-16/12] [label] full reformat disk!!!!!
sect         [hex no.] display sector
help         Display menu specific help
quit         Leave command mode
>re-format TEST
Formatting FAT32 in progress - please wait...
D:\>**Disk formatted
Disk D mounted
dir
Directory D:\

0 files with 0 bytes
0 directories, 4115947520 bytes free
D:\>info

Disk TEST (4124049408 bytes) SDHC FAT32
Bytes per sector: 512
Cluster size: 4096
Directory base: 0x00000002
FAT start: 0x0000005f
FAT size: 0x00001eb3
Number of FATs: 2
LBA: 0x00003dc5
Total clusters: 0x000f5547
Info sect: 0x00000040
Free clusters: 0x000f5546
Next free: 0x00000003
CSD: 0x40 0x0e 0x00 0x32 0x5b 0x59 0x00 0x00 0x1e 0xb9 0x7f 0x80 0x0a 0x40 0xff 0x00
D:\>sect 3dc5
Reading sector 0x00003dc5
 0x54534554 0x00000000 0x08000000 0x60000000 0x44eb44eb 0x60000000 0x000044eb 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
D:\>mkdir dir1
sect 3dc5
Reading sector 0x00003dc5
0x54534554 0x00000000 0x08000000 0x60000000 0x44eb44eb 0x60000000 0x000044eb 0x00000000 0x69006441 0x31007200 0x0f000000 0xffffe400 0xffffffff 0xffffffff 0x0000ffff 0xffffffff
 0x0a010020 0x0c161d0c 0x1000002f 0x60000000 0x44eb44eb 0x60000000 0x000344eb 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
D:\>

Notice that in the root directory of a freshly formatted card there is just a single entry (the volume entry - TEST in this case). When I create a directory there is a new directory object added immediately after it.
From what I could see from your card, each new directory object is added in the next cluster (4kBytes away), rather than "immediately" after the present entry.

In fnCreateFile():
    ptrFoundEntry = (DIR_ENTRY_STRUCTURE_FAT32 *)ptr_utDisk->ptrSectorData; // the directory entry in the sector buffer
    ptrFoundEntry += ptrDiskLocation->ucDirectoryEntry;                  // move to the present directory entry

Here the pointer ptrFoundEntry should point to the first entry and then increment to the next (free location). ptrDiskLocation->ucDirectoryEntry will be 1 on first use and so increment by 32 bytes.

If this works correctly the question is whether it is a problem that starts only over time and usage.

Regards

Mark

[neil]

Hi Mark
  Thanks for this. Where would I find the latest version of your utasker Demo?  It happens on all cards we used, and as soon as we use them.

Regards
Neil

[mark]

Neil

If you are using debug.c you will have this interface in your code.
You can take V1.4.7 and build for a standard target - or better, the latest developer' code at http://www.utasker.com/kinetis/developers.html

If you would like to compare with a standard build there are binaries at http://www.utasker.com/kinetis.html

Regards

Mark