bigger files with uFileSystem

[mvaurelien]

First of all, I've read "uTaskerFileSystemSPI_FLASH_003.pdf", "uTaskerFileSystem_3.pdf", "FileSystemLPC23xx_336k.doc", and "FileSystemLPC23xx.doc". But there is something I'm still not understanding.
According to "uTaskerFileSystem_3.pdf", by specifying MAX_FILE_LENGTH as an unsigned long, we allow individual files larger than 64k. Then in "FileSystemLPC23xx_336k.doc", you say that "Files of multiple 4k lengths can also be saved. The final file ‘z’ can have a length of up to 136k".

But when I launch simulation, if I try to send by FTP a 128kB file "z.HTM", it failed (simulator is crashing). On my target (MCB2300 KEIL), it looks better. 137kB HTML file is quickly sent and reachable by HTTP. So it was just a bug on the simulator.

Now, I want to use my 4MB SPI flash to store and set 2 files of 1 MB. At the moment, I can't use the simulator anymore and I'm going to try with my MCB2300 and a 1MB SPI flash (AT26DF081).

So, concerning external SPI flash and according to "uTaskerFileSystemSPI_FLASH_003.pdf" :
- uFileSystem allow up to 64 files
- SINGLE_FILE_SIZE is a multiple of FILE_GRANULARITY (4096 bytes granularity in my case) and could be set to 64*FILE_GRANULARITY to allow 256kB file.
- FILE_SYSTEM_SIZE could be set to (2*SINGLE_FILE_SIZE + PAR_BLOCK_SIZE) to reserve 512MB for file system (plus parameter blocks).
In your opinion, could it work ?

Thanks, I'll give you results after performing some tests.

[mark]

Hi

"Files of multiple 4k lengths can be saved". This is special for the last sectors in the LPC23XX FLASH (not for the main ones, which have a granularity of 32k). The uFileSystem handles the sectors automatically and so allows a file (at the end of FLASH) to also occupy them if necessary.

When working with the simulator make sure that the simulated FLASH is blank before performing your test (delete FLASH_xxx.ini in the simulator directory). Depending on how the data is being saved, it may be that a FLASH programming rule is being broken (see the following for details about the special properties of the NXP FLASH: http://www.utasker.com/forum/index.php?topic=136.0)

Check also whether the simulator is generating an exception using code like *(unsigned char *)0 = 0, and you may see a comment as to the reason. It sometimes generates an exception to point out that the FLASH may have been used incorrectly and it could thus fail on the target. Be careful with the fact that the uFileSystem uses large granularity mode (see: http://www.utasker.com/docs/STR91XF/FileSystemSTR91X.PDF) which needs to be used with some care!

If the simulator crash can not be explained as above it may be a bug (possibly requiring a certain set up to see) so it would be useful to cure it.

You should be able to test operation with SPI FLASH in the simulator (at least with ATMEL FLASH), which would also allow you to verify the configuration. Don't forget to enable BOTH SPI_FILE_SYSTEM AND FLASH_FILE_SYSTEM for this work. Using a 1MByte SPI FLASH chip (note that an AT45DB081 driver is supported but not the AT26DF081 - I believe this has different commands) it is possible to define 64 files of 16k FILE GRANULARITY. The file granularity can be a multiple of the FLASH GRANULARITY, so this may the be best configuration for you. As you known, one file can occupy also multiple file spaces, so you could still decide to save just 2 files of 512k each.

The uParameterSystem can be put to the last sectors of internal FLASH in order to keep the SPI FLASH for only uFileSystem data.

If you already know that you will never need more that 2 files it is also possible to to define 2 files or 512k FILE GRANULARITY (the granularity simply has to be a multiple of the physical FLASH granularity).

Generally the uFileSystem has proved to be very flexible in a number of project up to 32Meg SPI FLASH (which requires multiple SPI FLASH). Once you have the configuration right you shouldn't have any problems - also in the simulator this should work correctly.

regards

Mark

[mvaurelien]

Mark,

uTasker is great ! I need to learn a lot about it but I succeed in using an external flash (I found a STM25P80) with my MCB2300 for storing big files. And I used telnet as a debug output. Amazing !

Now I want to use an UART as a debug output (I failed this evening) and use another SPI flash.

Sometimes I can't delete files on my SPI flash. Do you know this problem ?

Anyway, thank you so much.


Aurélien

[mark]

Hi Aurélien

It is good to hear that you made some useful progress.

The demo project uses a UART as debug output (set the one to used with DEMO_UART in app_hw_lpc23xx.h). Also check the following for details for using UARTs:
http://www.utasker.com/forum/index.php?topic=54.0
http://www.utasker.com/forum/index.php?topic=54.0

Note that in the demo project either TELNET or UART can be used for debug output. When one is active it will block the other. However you can still send debug output to BOTH Telnet (when connected) AND a UART by making the following small change:

In debug.c

// Original - debug output sent to Telnet
extern QUEUE_TRANSFER fnNetworkTx(unsigned char *output_buffer, QUEUE_TRANSFER nr_of_bytes)
{
      return (fnSendBufTCP(Telnet_socket, output_buffer, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY)));
}

Code: [Select]

// Modified - debug output sent to UART and Telnet
extern QUEUE_TRANSFER fnNetworkTx(unsigned char *output_buffer, QUEUE_TRANSFER nr_of_bytes)
{
    fnWrite(SerialPortID, output_buffer, nr_of_bytes);
      return (fnSendBufTCP(Telnet_socket, output_buffer, nr_of_bytes, (TCP_BUF_SEND | TCP_BUF_SEND_REPORT_COPY)));
}
If a Telnet connection exists this is the debug output patch used. It will first send to the UART (a handle to USB could also be used, or multiple UARTs, etc.) and then to TELNET.

I don't know of a reason why the SPI FLASH would not always be deleted. If you don't find the cause you could give some more details so that I could look a bit closer.

Regards

Mark


 

[mvaurelien]

ok Mark, it was my mistake. UART is ok now and storing a 980kB file in my 1MB SPI flash seems to be ok.

Here is my problem : my configuration (with my STM25P80 code flash) should allow me to use 16 files up to 65536 kB or 2 files up to 512k. But, when I try to copy on the FTP the 9 webpages, it only keep the last 5. When trying to copy 2 files of  504k, it only keep the last one.

Here is my configuration :

in config.h:
-#define SPI_FILE_SYSTEM
-#define FLASH_FILE_SYSTEM
-#define SPI_FLASH_ST
-#define FILE_GRANULARITY (SPI_FLASH_BLOCK_LENGTH)

in app_he_lpc23xx.h :
--#define SPI_FLASH_SECTOR_LENGTH (256*SPI_FLASH_PAGE_LENGTH) //code flash and not data flash
-#define SPI_FLASH_BLOCK_LENGTH  SPI_FLASH_SECTOR_LENGTH //64kB
-#define SPI_FLASH_START        (FLASH_START_ADDRESS + SIZE_OF_FLASH)
-#define PAR_BLOCK_SIZE  (2*PARAMETER_BLOCK_SIZE)
-#define uFILE_START (SPI_FLASH_START - PAR_BLOCK_SIZE) //param in internal flash
-#define SINGLE_FILE_SIZE (FILE_GRANULARITY) //64kB
-#define FILE_SYSTEM_SIZE (16*SINGLE_FILE_SIZE+ PAR_BLOCK_SIZE) //16*64kB=1024kB=1MB reserved for file system (plus parameter blocks)

Am I missing something ?

Thanks.

[mark]

Hi

This looks good to me.

You didn't mention how the files are named.

If you want to store 2 files of about 512k (don't forget the small header overhead of about 5 bytes!) they need to be named '0' and '8'. Then they will fit correctly.

If you, for example, save a 512k file '0' it will be saved at the start of flash. If you then save a 512k file '1' it will be saved starting in the second sector and 'bull-doze' any existing files out of the way. After the copy you will only have the file '1'.

Is that your problem? Check the naming convention and maybe it will work as required.

regards

Mark

[mvaurelien]

that's what I was missing ! If I'm right : 62 files are supported with names from "0" to "9" then from "A" to "Z" and then from "a" to "z". And in each file, it is necessary to keep 5 bytes free for header overhead, isn't it ?

Everything is ok, now I'm going to use my ATMEL SPI flash.

Thanks again.