µTasker Forum
µTasker Forum => utFAT => Topic started by: mark on December 12, 2009, 12:11:20 AM
-
Hi All
Many users have asked about it and its preparation is nearing completion, therefore it is time to present some first details.
The utFAT (uTasker FAT) is FAT32 compatible and will soon be available for use with SD-cards. It supports LFNs (Long File Names) - which can also be disabled. It supports standard and high capacity V2 SD-cards.
It doesn't support FAT12/16 and also not older MMCs. However the difference is in fact quite small and could be added if there is real demand. The idea behind this simply being that MMCs and FAT12/16 no longer correspond to the present state of technology and so it seems a nice chance to start with that which is relevant today rather than dragging around also remnants of bygone times.
It has a simple and practical user interface. This was not copied from standard interfaces but chosen based on practicality - the calls enclose functions which do things that users will hopefully want to do in real-world application in an encapsulated and logical manner. For example, there is no interface for mounting disks - this is fully automated by the underlying mass-storage task and so, if the SD-card is inserted, it can be used, if not the disk drive is not available... the first SD-Card appears as drive D:/
When no SD-card is available fall-back to the uFileSystem is possible.
It includes an interface including the detailed work to do typical user interface things. For example, to add a DOS type user interface (which is included in the debug task) the code required doesn't need to get involved with how the directory paths work since this is all encapsulated in a couple of easy to use functions. The demo allows this type of interface to operate on the UART, via TELNET and/or via USB. See an example at the bottom.
It is fully integrated into the FTP server, allowing browsing through directories, retrieving files etc. The access point for the FTP server within the directory structure can be set in case such accesses are to be restricted to certain directories on the SD-card.
It is fully integrated into the web server, allowing complete existing web server content to be copied from a PC to an SD card and immediately be served from an embedded web server. The location of the HTTP server's root directory and its default file can be configured.
When working with LFNs the Linux workaround technique can be optionally used to (presumably) avoid potential patent issues in commercial projects.
The uTasker simulator operates with a simulated SD card making testing, debugging and study of the utFAT - and FAT32 in general - very comfortable.
RAM use for single-user FTP and 6 parallel HTTP sessions with LFN support is about 700 bytes. Users interfaces (like DOS type command line) need a complete directory path, which can be long with LFN and deep directories, but is user configurable to suit the application, therefore 300 byte or so is often adequate.
Lots more technical details be be disclosed as the package is finalised....
That was a first brief presentation, which is hoped to wake interest in some of the new opportunities that it adds to the project use. A first release is planned for the end of the year so I am looking forward to some interesting discussions and then some interesting projects ;-)
Regards
Mark
Below is an example of the DOS-like command line interface in action (it is in fact a copy of the complete uTasker web site on an SD-card. All standard web content is displayed by the processor's web server as if it were the real web site...):
Disk interface
===================
up go to main menu
dir [path] show directory content
cd [path] change dir. (.. for up)
prt [path] print file (ASCII)
root set root dir
del [path] delete file or dir.
help Display menu specific help
quit Leave command mode
>dir
Directory D:\
---A 23.11.2009 22:31 9 Test1.txt
---- 23.11.2009 22:31 <DIR> DIR_1
---- 28.11.2009 14:13 <DIR> uTaskerWeb
---- 29.11.2009 17:13 <DIR> webPages
1 files with 9 bytes
3 directories, 1496460228 bytes free
D:\>cd uTaskerWeb
D:\uTaskerWeb>dir
Directory D:\uTaskerWeb\
---A 02.10.2009 15:13 6442 avr32.html
---A 03.06.2009 02:43 12026 avr32_1.gif
---A 03.06.2009 02:35 42176 avr32_2.gif
---A 12.05.2007 15:31 15545 bd-cowo-blumen.gif
---A 06.09.2006 22:48 2007 BLogo.jpg
---A 05.04.2007 20:08 13832 chip1.jpg
---A 05.04.2007 20:11 5325 chip2.jpg
---A 05.04.2007 20:12 3369 chip3.jpg
---A 05.04.2007 20:12 2275 chip4.jpg
---A 05.04.2007 20:12 1461 chip5.jpg
---A 12.08.2008 17:18 319367 ColdFire_Graphic_2.jpeg
---A 27.09.2006 23:36 20992 EBV.doc
---A 13.04.2009 20:04 318 favicon.ico
---A 02.10.2009 15:13 5963 forum.html
---A 19.06.2007 20:03 413 ie-navi.css
---A 02.10.2009 15:18 21221 index.html
---A 23.03.2008 00:29 17790 index_.html
---A 02.10.2009 15:13 5768 kirin3.html
---A 22.10.2009 19:38 9539 links.html
---A 02.10.2009 15:13 7788 modbus.html
---A 01.08.2009 01:51 59615 modbus1.jpg
---A 01.08.2009 02:03 65343 modbus2.jpg
---A 12.05.2007 15:56 1509 muttertag.html
---A 19.06.2007 19:51 171 navi.css
---A 22.01.2009 16:30 3731 New.gif
---A 26.02.2007 12:30 326904 PIC_0049.JPG
---A 26.02.2007 12:31 342977 PIC_0050.JPG
---A 26.02.2007 12:32 379115 PIC_0052.JPG
---A 02.10.2009 15:13 15636 support.html
---A 28.11.2009 14:13 23844 SW_Demos.html
---A 06.09.2006 10:38 1719 Text1.txt
-HSA 20.01.2008 00:36 85504 Thumbs.db
---A 04.09.2006 21:38 55038 uTasker.bmp
---A 14.09.2006 16:37 69632 uTasker.doc
---A 04.09.2006 21:17 14865 uTasker.JPG
---A 14.09.2006 16:51 52990 uTasker.PDF
---A 04.09.2006 21:17 128390 uTasker2.bmp
---A 14.04.2009 18:21 1673 uTaskerAv.jpg
---A 14.04.2009 18:30 1669 uTaskerAv1.jpg
---A 04.09.2006 21:48 35887 uTaskerLogo.jpg
---A 07.10.2006 13:41 12971 uTaskerLogoR.gif
---A 04.09.2006 21:48 24610 uTaskerLogoR.jpg
---A 07.10.2006 13:43 11759 uTaskerLogoR1.gif
---A 19.11.2006 18:38 2634 uTaskerLogoS.jpg
---A 03.11.2006 02:07 4943 uTaskerLogoSmall.jpg
---A 06.02.2007 20:24 1600 uTaskerLogoSS.jpg
---A 07.10.2006 13:47 55654 uTaskerR.bmp
---A 07.10.2006 13:50 2498 uTaskerR.GIF
---A 27.09.2006 16:33 69632 uTasker_d.doc
---- 02.10.2009 15:13 <DIR> Demos
---- 11.10.2009 19:11 <DIR> docs
---- 06.02.2009 21:32 <DIR> Forum
---- 02.10.2009 15:13 <DIR> Licensing
---- 28.10.2008 16:52 <DIR> mjb
---- 02.10.2009 15:12 <DIR> Payment
---- 27.10.2009 00:49 <DIR> Projects
---- 02.10.2009 15:13 <DIR> software
---- 18.01.2009 21:36 <DIR> temp
---- 02.10.2009 15:23 <DIR> test
49 files with 2366130 bytes
10 directories, 1496460228 bytes free
D:\uTaskerWeb>cd demos
D:\uTaskerWeb\demos>dir
Directory D:\uTaskerWeb\demos\
---A 20.01.2009 22:48 109387 Kirin3.jpg
---A 02.10.2009 15:13 6027 Simulations.html
-HSA 20.01.2008 00:36 30720 Thumbs.db
---A 10.06.2009 00:56 68503 uSimAVR32.JPG
---A 31.07.2009 16:08 64484 uSimAVR321.JPG
---A 10.03.2009 23:05 50316 uSimLM3S1XX.JPG
---A 08.04.2009 22:43 46866 uSimLM3S6XXX.JPG
---A 02.12.2007 22:47 41017 uSimLM3S6XXX_old.JPG
---A 11.11.2007 23:59 40090 uSimLM3S6XXX__.JPG
---A 02.01.2007 18:53 48967 uSimLPC23XX.JPG
---A 21.06.2009 14:36 74857 uSimLPC24XX.JPG
---A 14.09.2008 19:49 47438 uSimM52211.JPG
---A 02.01.2007 18:53 47815 uSimNE64.JPG
---A 02.01.2007 18:53 44467 uSimSAM7X.JPG
---A 02.01.2007 18:53 51307 uSimSTR91XF.JPG
---A 10.01.2007 22:16 85076 uTaskerSim.JPG
---- 14.07.2008 16:36 <DIR> exceptions
---- 21.11.2009 15:59 <DIR> exes
---- 14.07.2008 16:36 <DIR> Freescale
---- 21.01.2009 21:25 <DIR> M522XX
---- 14.07.2008 16:36 <DIR> Rabbit
---- 03.01.2009 23:13 <DIR> SAM7X
16 files with 857337 bytes
6 directories, 1496460228 bytes free
D:\uTaskerWeb\demos>cd ../../webpages
D:\webpages>dir
Directory D:\webpages\
---- 16.10.2009 14:07 <DIR> WebPagesLPC23xx
---- 02.08.2009 19:52 <DIR> WebPagesM5223X
---- 16.10.2009 16:01 <DIR> WebPagesM5225X
---- 20.06.2008 13:50 <DIR> WebPagesNE64
---- 29.07.2009 13:39 <DIR> WebPagesSAM7X
---- 21.05.2009 19:17 <DIR> WebPagesSTR91XF
---- 29.07.2009 21:02 <DIR> WebPagesAVR32
---- 31.10.2009 17:21 <DIR> WebPagesLM3Sxxxx
0 files with 0 bytes
8 directories, 1496460228 bytes free
D:\webpages>
-
Mark,
uFat looks to be very interesting, I have a few of questions
1. Will it be possible to run a uFat with an SPI memory chip instead of an SD card? I'm interested because I would like to use a slightly more flexible file name scheme rather than relying on the first letter dictating the file location
2. Will uFat be able to support userfile collections stored as a single file in the uFat system?
3. Will uFat support similar commands to the uFileSystem making porting easier?
Cheers
Martin
-
Hi Mark,
It looks nice.
Do you plan to allow simultaneous access to a file (one write access with multiple read access)?
Do you have a pre-release available at this time?
-
Hi Martin
1) SD cards and SPI FLASH are quite different due to the fact that SD cards have a controller in them handling such details like level-wearing and bad block management (this is invisible to the user but is probably highly developed) - at least I think... (I read a SanDisc technical document where this point seems to be confirmed). Therefore level wearing is something that would be quite difficult to retain. This is however not a problem in applications where the quantity of writes is not too high.
Also the dimensions of SPI based FAT32 may call for a slightly different configuration. ATMEL SPI FLASH chips (with small granularity) seem to me to be good candidates since they fit well, whereas large granularity types will be a bit of pain.
Although I do see that FAT would fit certain SPI FLASH devices fairly well it would still mean a quite different hardware interface to the SD card one. If the demand is there it can be looked at after the SD card work is completed and established.
2) I don't know that there would be any advantage of a user file collection stored in the utFAT. It could also not operate with the existing addressing mechanism. [Reminder: SPI FLASH memory is effectively linear and so can be memory mapped (virtually, as the uFileSystem does). SD cards - and FAT in general - are not linear, they work with sectors (effectively windows into a small block of memory) and and clusters which are like blocks of the files (or directory information) linked together almost anywhere in the memory (not linear)] If utFAT is used I think that standard file addressing is the way to go. However if SPI FLASH is used I find embedded user files a good compromise to allowing it to essentially look like a file system with long file names and even directories (the complete directory path is the file name), complete web pages to be packed into one single file for uploading and fast access (still not as fast as uFileSystem access but faster that FAT32 access since the file search is generally rather less complicated).
3) No, it is not really possible to use compatible commands. The operation of the two are just so different. However it is still easy to use. Here is an example (present state and may change before release...Also no error handling for simplicity.)
FTP file retrieval, uFileSystem case - the file is identified from the path/name. The length is checked and,
ptrFile = uOpenFile((CHAR *)(ucIp_Data+5)); // get file pointer
if ((uGetFileLength(ptrFile)) && (fnTCP_Connect(FTP_TCP_Data_socket, ipFTP_data, usFTP_DataPort, FTP_DATA_PORT, 0) >= 0)) {
ucFTP_action = DO_UPLOAD;
FileOffset = 0; // start at beginning of file
}
.. data is put to the output buffer.
uGetFileData(ptrFile, FileOffset, ptrBuffer, usTxLength)); // read from the file to a buffer
.. (with some management of the length and file offset)
In the utFAT case:
static UTDIRECTORY utDirectory = {0}; // directory object for a single user
static UTFILE utFile = {0}; // file object for a single connection
utOpenDirectory(DISK_D, FTP_ROOT, &utDirectory); // set the FTP root (eg. D:/ftp_dir/) - initialisation
utFile.ptr_utDirObject = &utDirectory; // link the file object to its directory object
.. locate the file according to the retrieval name (eg. /dir_1/file_000000012888_long_name.txt
utOpenFile((const CHAR *)(ucIp_Data+5), &utFile, UTFAT_OPEN_FOR_READ);
..
if (utFile.ulFileSize != 0) {
utReadFile(&utFile, ptrBuffer, usTxLength); // read from the file to a buffer
}
In this case the management of the file is content is performed in the utReadFile() (and UTFILE object) since it needs to be able to do seek type functions.
Integrating this into FTP and HTTP was not difficult. In fact, since the utFAT (necessarily) performs extra file location management some of the pointers/indexers used together with the uFileSystem are superfluous. I just set them to the momentary utFAT values to keep them in sync so that it is easier to understand the two when they are side-by-side. Eg.
FileLength = utFile.ulFileSize; // set the local length (as used by uFileSystem code) to the internal value to ensure compatibility with later length use
In an attempt to ensure porting (or using both files systems together - useful as a fall back in case the card is extracted...) I have encapsulated details of mounting disks. The only addition code is setting where the directory is located (can also be root), which initialises everything for its subsequent use by the particular user (see utOpenDirectory()) - whereby there may need to be be multiple directory objects if multiple users have their own relative locations in the utFAT (like multiple DOS windows each have their own independent present location). Then it is easy to move to other directory locations - allowing absolute or relative addressing (relative is generally more efficient for the file searches too). The user code shouldn't need to worry about too much detail (eg. - D:/dir_2/myFile.pdf or ../../dir_3/dir_4/myOtherFile.jpeg - basically just pass an appropriate string).
Regards
Mark
-
Hi Hervé
File sharing for reads is certainly possible. At the same time I am working on additional file sharing management (also for the uFileSystem), where users can protect files from deletes and modification while they are being used.
I expect that this mechanism can be shared by all file system types.
At the moment I don't have a pre-release that I have made available due to the fact that there is still a lot of activity on it (interfaces change as further tests and experience show that the solution still needs some optimisation). The goal is a first release around Christmas and pre-releases would greatly endanger this - this would be supported by all processors (this needs some work since the SPI interfaces have to be adapted for each) but not have all features. At a minimum all read support will be available but not file sharing management (also not necessary when only read) and not (all) write [delete, create, format, etc.] capabilities. The last bit is left open depending on the progress, but basically it is seen as important to have a well defined interface supporting all read requirements as step 1; if this foundation is solid the second phase of adding write support and other extensions will be easier. Note that some good progress has been made and so I do hope to have some write capabilities in the first release but I am not committing to it, in preference to a mature solution in the long run.
Regards
Mark