SD Card questions

[dkg]

Hi Mark,

I now have the uTasker SD Card support working on our hardware. It works well but I found a few things lacking before we could use this in our product. I was just wondering if you have plans to implement (or have already implemented) the following features:

1) Card insertion/removal detection
2) Honor the mechanical write-protect switch on the SD card
3) Supporting both the SD Card and internal flash file system concurrently (separate drives maybe?)

For the first two items, there are signals from the SD Card socket that provide this information. Right now, I noticed things get a little wonky if I remove the SD Card and re-insert it without resetting the application. Also,  I think our customers might be upset with us if we allowed them to write to an SD Card that was supposed to be locked with the mechanical write-protect switch.

I also hope to implement a USB mass storage class to support file management on the SD Card installed on the product. I don't want to expose our customers to the debug command shell and it would be a natural to allow their host operating system to manage the files just like a USB drive.

One other thing that may be a problem is the lack of LFN support for creating/writing files. I read your document and believe I understand the reason for this limitation. I need to investigate the issue further and explain it to my team... or implement it if we feel we can support it.

Thanks,
Dave G.

[mhoneywill]

Hi Dave,

I too hit your point1 here is the text from an email I sent to Mark about this, it also gives a possible solution to detecting a changed card by using its serial number.

Point 2, I would think would be easy to add, not sure about point 3

Thanks Mark,

I understand about the switches, it would be good not to have to use them. But it would be useful to detect card changes, I can’t guarantee what the user will do.

Do SD cards have unique serial numbers that could be checked? A quick google seemed to talk about the SD cards serial number stored in the CID register, could this be polled periodically to check if it exits / has changed, which would indicate a change?
Page 60 of this document http://www.sdcard.org/developers/tech/sdcard/pls/Simplified_Physical_Layer_Spec.pdf  talks about CMD10 being used to read the CID of a card.

With your demo is there any command to re-mount the card?
Maybe the info command could read the CID and remount the card if a different CID is found

FAT16 support would be nice, as I can see that some user devices would be preformatted this way. I agree with you about FAT12. I presume you would support read and write under FAT16.

Just compiled V1.8 and confirmed it works as before

Would it be helpful to transfer this conversation to the forum, for the benefit of other users or just post a summary at the end?

Cheers

Martin

From: Mark Butcher [mailto:Mark@uTasker.com]
Sent: 05 January 2011 13:45
To: Martin Honeywill
Subject: AW: Testing utFAT - more stuff

Hi Martin

I attached a V1.8 (it doesn’t have anything that will change things – just NAND FLASH interface and a couple of minor features as in the thread).

The problem is the following: SD card insertion and removal is detected by switches on the SD card sockets. The Luminary board has these switches connected to GND so they can’t be used (also the write protect switch cannot be detected). Some of the sockets also don’t have these switches physically. 90% of boards don’t connect them and so I didn’t put any generic support for them in the code for this reason.

Detection is achieved the first time due to the fact that the interface is regularly trying to mount the card. When a card is inserted it can be mounted and subsequently used.

If the card is removed the SW doesn’t know this. It will be detected only the next time that accesses are attempted, which fail. Then it would be possible to restart the mounting/detection process if required. If the mounted card is removed and a different one inserted this will not be seen. It can result in strange effects if files are open and especially if the card is different.

That means that it is essentially designed for a fixed inserted card – mainly since there is often no switch available (as on the LM boards).

Only FAT 32 is supported – in fact I will add FAT 16 soon since I don’t think that Windows likes working with FAT32 when the file system is small (eg. When a small NAND Flash is used). It should only be a few lines of code. I don’t think that FAT12 is of any use though (it is more complicated and FAT16 seems small enough anyway for typical FLASH chips).

Regards

Mark

[dkg]

Hi Martin,

Thanks for the info. I am not sure a serial number check would be sufficient since I see problems removing/re-inserting the same card -- it would have the same serial number. The card apparently needs to be remounted/re-initialized in this situation. This might be a simple fix without using the card present signal by keeping track of when the card responds and doesn't respond (and maybe that is what Mark was talking about but hasn't implemented?)

On page 32 of the Simplified Physical Layer Spec v3.01 under the section titled "Mechanical Write Protect Switch" it says "It is the responsibility of the host to protect the card." I don't see how that can be done without knowing the status of that mechanical switch.

Although some hardware apparently doesn't have those two signals, I think the software should support it for those who have them (which I would think is the majority of hardware designs?).

Dave G.

[mhoneywill]

Hi Dave,

I agree about the write protect switch it should be supported in software and just #defined out if not used.

As you say, a system that will re-init the card if it detects a no reply would be nice and neat as it would automatically cover both cases of a reinsertion of the same card or a re-insertion of a different card.

I'm presuming a newly inserted card will not communicate unless an init sequence has been performed.

Also LFN support when writing would be nice.

Cheers

Martin

[mark]

Hi Dave, Martin

Just a quick overview:

- presently I am working on FAT16 support since it has turned out to be necessary to get Windows XP to co-opertae with NAND Flash smaller that 32M (Vista and Win 7 are OK working with FAT32 even in such small memory but it will be necessary to have XP working with it too for some time).

- I was suprised at how few SD card holders have switches integrated into them. Some which do, just had them shorted to ground in the boards and so I hardly had a development board which allowed them to be used, so I left this out. However the utFAT does have a flag (utDisks[ucDrive].usDiskFlags & WRITE_PROTECTED_SD_CARD) which can be set by any code that is capable of reading the switch state of the write protection. It actually only blocks formatting (not writing or deleting, for some reason) at the moment but this will be quite easy to add at the start of any routines that attempt to write.
In may cases the application code, which calls writes, deletes etc. can also check the switch and avoid calling these (rather than calling and checking the return value to see whether it failed due to this). Write protected files on a card should already be protected (due to their read-only state in the file attributes).
Finally there is also a write proect setting in the SD card registers themselves. I didn't try using this though, and don't actually know how it gets set - possibly requires a special utility for PC user's to actually set it themselves(?)

- The utFAT module is about 1 year old now. In its first version it has a few constraint (like not being able to manage cards being removed and re-inserted during normal operation) and it generally does takes a year or so of first users doing real work with it to identify in which direction further development should be channeled (this is also quite typical of introduction times for new processor packages too, before real work begins to be done with them).

- There is a lot that can be done with file systems like this; USB MSD is available in newer Beta versions - as SD-card and as NAND Flash - these are being refined in first projects and maybe a number of LUNs (the logical units; eg. SD card seen as Disk E:, internal Flash as disk F:, external SPI Flash as disk G: and external NAND Flash as disk H:) will be possible with time - it does however take quite a lot of development effor to get that far though...)

- It is already possible to use SD-card (utFAT) and uFileSystem (internal FLASH or external SPI Flash) at the same time. Not yet to have these two areas seen as different FAT disks, but utFAT is basically build so that it should be possible (with corresponding development effort).

- Since SD cards are often shared between embedded system and PCs I decided that LFN would be essential, at least being able to read them in the embedded system.
The compilcations with writing (and deleting) are:
- there is the patent issue when writing LFN and probably the Linux workaround would have to be used (presuming that this doesn't get involved in a court-case in the near future). Such a problem would effectively mean any LFN write users would suddenly have potential problems too. Playing safe, it was not done (also it made life simpler for the first version). Technically it is however not that difficult but small embedded systems will (in my opinion) usually be able to live with writing 8:3 format without big restrictions.
- the utFAT can not delete LFN due to the fact that LFN are stored in multiple clusters. Reading them is no problem but deleting them requires working through clusters in the reverse direction. Clusters are easy to search in the forward direction but haven't any support for searching in the reverse direction apart from simple brute-force (meaning lots of search time). Windows actually reads the complete FAT from the card as first job when it connects and so has the advantage that the FAT is in local RAM, where it can of course search quite fast - embededed systems don't usually have the luxury of being able to cache the FAT (it would typically need several MBytes of RAM to do this) and so has to search by reading the card (which is slow) and may need to actually first search several MByte of data to be able to delete a LFN....for the first instance I decided to avoid this complexity. Of course, with the right motivation (and time and test projects) there is nothing to stop us performing the next steps and also achieving this.

- I do see NAND Flash based FAT as potentially very interesting in the near future - more and more single-chip processors have NAND Flash controllers in them (like the new Kinetis and STM devices, as well as established SAM7SE). The complication is that the processor needs to do memory management (mapping virtual sections to physical memory) and also level-wear (ensuring that the write/deletes are spread across the phyical memory area to avoid wear-out of high-useage sectors) and also bad-block management (NAND is not 100% reliable and so needs to error checking and management of sectors that fail early or are also defect on delivery - the manufacturer guaranties that there are a minimum number of operational sectors and not that they all work).
This is quite complicated (especially to get 100% right) but a new document will be appearing soon on the doc page detailing exactly the solution used - adapting to different chips is then quite easy.

Regards

Mark

[dkg]

Hi Mark,

Thanks for the update. I would've been right there with you a year ago working on utFAT except that marketing had other ideas of what my priorities should be

When I was referring to using both utFAT and uFileSystem concurrently I was actually talking about accessing both of them through FTP.  Maybe that is possible already but I overlooked something? BTW, I don't need or want FAT on the internal filesystem.

I believe I have run across something online about a product out there that used the internal write-protect feature on an SD-CARD. In fact, I think it was a cell-phone and a microSD card. It was something that no one really uses and no one wants to use since the OS support wasn't there. I remember seeing someone have a problem with getting the card in that state and then unable to use it with Windows. In any case, that is not what I want at all. What I think is important for our customers is to support the lock/write-protect switch on the SD-Card itself which has an entirely different purpose than individually write-protecting files on the card.

LFN is important for the reason you gave -- we expect the SD Card to be used directly on a PC as well as our embedded product and to transfer data between them. I don't know yet how important it is for the product itself to create/writing LFN files. I can see how customers might think an 8.3 restriction is so "'90s" and it might shed a negative light on the product (not being up to date with technology?). However, it all depends on how we expect our customers to use the card and we haven't fleshed that out yet.

Dave G.

[mark]

Dave

FTP supports both the uFileSystem and utFAT, however it checks first whether there is an SD card present and uses this if it is, or falls back to the uFileSystem if not. This doesn't allow both to be used at the same time.

However I expect that it should be quite easy to allow switching to the uFileSystem by entereing something like "cd E:\" - as long as such a directory path can be used to control a flag ("cd D:\" to switch back to the SD card) it will the be possible to access the file system selected (rather than based on whether an SD card is mounted, it checks a different flag).


For the case of write protection of cards via WP switch I have prepared the following (its is performed when the SD card is mounted):

            if (GET_SDCARD_WP_STATE()) {
                utDisks[0].usDiskFlags |= (DISK_MOUNTED | WRITE_PROTECTED_SD_CARD); // the write protected disk is now ready for use
            }
            else {
                utDisks[0].usDiskFlags |= DISK_MOUNTED;                  // the disk is now ready for use
            }


The user simply has to supply the macro GET_SDCARD_WP_STATE(). It can be 0 if no switch is used.
If the switch is connected to a port something like
#define GET_SDCARD_WP_STATE()   _READ_PORT_MASK(UC, SDCARD_WP_SWITCH)
can be used,  where the SDCARD_WP_SWITCH is the port bit (assuming that a '1' is detecting the WP switch active).

At a few locations where the user tries to delete, write, rename etc. the following new check then ensures that such write type operations are rejected:

    if (ptr_utDisk->usDiskFlags & WRITE_PROTECTED_SD_CARD) {
        return UTFAT_DISK_WRITE_PROTECTED;
    }


Once I have completed the FAT16 support that I am working on I will send you the update for validation.

Regards

Mark

[dkg]

Hi Mark,

Thanks for the info. Yes, I was planning on having the uFileSystem look like another drive as you suggest. I just didn't know if you had plans to support the changes I need

I look forward to testing your next update.

Dave G.