Bug reports: uMemcpy & RUN_LOOPS_IN_RAM

[Richard]

Hi, Mark.

In trying to understand parameter blocks, I seem to have run into a couple of bugs.

fnSaveNewPars(SAVE_NEW_PARAMETERS) calls (after some nesting) fnWriteLongWordFlash().  This relies on fnFlashNow().  Since it is generally not possible to execute code from FLASH when also writing to FLASH, the first time fnFlashNow() is used, it copies necessary code into RAM, using uMemcpy(), and then tries to execute the copy.

To copy more than 20 bytes (as defined by SMALLEST_DMA_COPY), uMemcpy() uses a DMA technique.  By setting breakpoints on the M52233DEMO board, I have found that in this case, the copy does not work; the relevant portion of memory contains mostly 0x00 after uMemcpy() returns.  If I set SMALLEST_DMA_COPY to some very high number, forcing a direct copy (*ptr1++ = *ptr2++;), uMemcpy() does the necessary copy and the expected code makes it into RAM.

In the process of working with this, I also tried commenting out DMA_MEMCPY_SET but defining RUN_LOOPS_IN_RAM.  This caused the compiler to complain a couple of times about an illegal constant expression.

I am using uTasker 1.3, SP6, and version 7.0 of the Codewarrior IDE, but I've tried this with version 6.4, too, and had the same results.

I am still having some difficulties understanding parameter blocks, but I'll post questions about those in the general forum when I have formulated my thoughts a little more sharply.

Best regards,
    Richard

[mark]

Hi Richard

First the RUN_LOOPS_IN_RAM issue.
This is not supported in the CW project from SP5 - here is the explanation in the release notes. Since the advantage of the loops is minor it is still on the 'to-solve' list. If you remove both DMA_MEMCPY_SET and RUN_LOOPS_IN_RAM then it will be the same as using standard memcpy().

1. The define RUN_LOOPS_IN_RAM is not supported in the SP5 under CodeWarrior. The reason is that the compiler won't accept the method of calcualting the code space when changing from malloc based to static RAM space, which is required due to new start-up requirements. The advantages of the loops in RAM are minor and DMA_MEMCPY_SET is recommended. Methods of solving this restrictions will be investigated.


Now to the more important DMA issue. Basically I am convinced that this works correctly because I know of many projects running with DMA support for copying memory - it is used in many places; user code, the TCP/IP stack and most queues (such as inter-task and UARTs). If the copy of the FLASH program were not to work any write to, or delete of FLASH would result in an immediate crash of the processor.

Therefore my assumption is that what you are observing has something to do with the operation of the debugger. But, I can't explan what...

When you are working with the parameter system and saving new data is it crashing when the define DMA_MEMCPY_SET is active? If it is not crashing (and the writing to FLASH is working) the copy must have worked correctly (even if it looks bad in the debugger). If it is really crashing, does it do this with the uTasker reference project (which is configured to use this in SP6) or in a different project?

Regards

Mark

PS. Note that the uParameterSystem and its use in the demo project is described in some detail in pages 9..13 of the following document:
http://www.utasker.com/docs/uTasker/uTaskerFileSystem_3.PDF

[Richard]

Hi, Mark.
Thanks for the prompt reply.
As for the RUN_LOOPS_IN_RAM issue, I didn't realize that it was unsupported.

As for the DMA problem, at least when running using the debugger and setting breakpoints, the code appears to hang, and when stopped manually is in an unimplemented interrupt routine, undef_int().  I did go back to a reasonably pure SP6 after seeing the problem with a version I've altered significantly.  It showed up in both.  By "reasonably pure", I mean that I've turned off DHCP, altered the IP addresses to be 192.168.1.x rather than 192.168.0.x, and changed the system to be the DEMO board rather than the EVB board.  To test for the DMA problem, I put a call to fnSaveNewPars(SAVE_NEW_PARAMETERS); right at the end of fnApplication().  One more thing: I've still not succeeded in running a pure SP6 using version 7.0, so in testing SP6, I reverted to version 6.4 of the IDE.

I have read version 2 of the discussion of the uTaskerFileSystem -- several times.  Does its explanation of the parameter system differ from version 3?

To keep the length of this message down, I'll attach my musings about the parameter system, which I wrote earlier today, as a text file.  If you like, feel free to answer off-line or to copy and edit the file and reply to the questions on-line, whichever you think is best for the forum.  I had intended to clean the text up a bit, but it seems more important to include it with this response, so please excuse its tendency to ramble.

Best regards,
    Richard

[mark]

Hi Richard

>>As for the DMA problem, at least when running using the debugger, the code appears to hang
Are you sure that the watchdog is disabled when debugging? - otherwise it fails when breaking and trying to step.

>>I've still not succeeded in running a pure SP6 using version 7.0
This is worrying me. I have tested SP6 also with CW 7.0 and couldn't find any problems. Also the DMA difficulty seems to be otherwise unknow.
Could you zip the complete project code and send it to me? Leave any target files (map and elf) as compiled by your V7.0 compiler so that I can run it on my demo board (I will not recompile for first tests and then recompile to see whether there is really a difference somewhere). Assuming that the DMA problem also exists in this project I can look also at that. To save space delete any VS targets (the debug directory that it creates) if you have been working with the simulator (the files there take up quite a lot of space).
I will send you an FTP address in a private mail so that you can send it easily.

>>Does its explanation of the parameter system differ from version 3?
No I don't think so - the document has been quite stable sind 5.1.2007 (there is always a list of modifications at the end and there is nothing listed with regards to the parameter system).

>>I'll attach my musings about the parameter system
OK. I look forward to your thoughts.

Regards

Mark

[Richard]

Hi, Mark.

>>As for the DMA problem, at least when running using the debugger, the code appears to hang
Are you sure that the watchdog is disabled when debugging? - otherwise it fails when breaking and trying to step.

I have never disabled the watchdog while debugging, and I've never had a hanging problem.  The only strange and annoying debugging behavior that I regularly see is that serial communication never resumes once the first breakpoint has been hit.  I've never bothered to try to figure out why, since if I'm single stepping, I am not usually making much use of the debugging data that I send out through the serial port.

To this, I must now add that further experimentation with uMemcpy() has shown the following weird debugger behavior that occurs if I single step through fnFlashNow() and then examine what is in RAM when fnRAM_code() is called.  If I do this the first time fnFlashNow() runs, it will not copy the fnRAM_code() routine into RAM, as I have already reported.  This happens even when the actual copy from FLASH to RAM is run at speed using run-to-cursor or some other way of executing the copy instructions.  However, if fnFlashNow() runs once before I hit a breakpoint, the necessary code is copied into RAM.  I confirmed this by adding two calls to fnSaveNewPars(SAVE_NEW_PARAMETERS), one right at the end of fnApplication() and another in my own routine that is called later.  If I wait until the second one is called and single step through fnFlashNow(), the fnRAM_code() routine has indeed already been copied to RAM.  This tends to confirm your comment

Therefore my assumption is that what you are observing has something to do with the operation of the debugger.

I will send you the requested copy of SP6 that hasn't worked with version 7.0 of the IDE in the next few minutes.  If you look in uTaskerSP6\Applications\uTaskerV1.3\CodeWarrior_M5223X\uTaskerV1.3, you'll find a .png file that captured the various updates that took place when first run under version 7.0 of the IDE.

Right now, I'm trying to figure out the maximum size that the PARS structure can have.  From the documentation, I thought the calculation would be 2046 Bytes (FLASH_GRANULARITY - 2) diminished by the size of the NETWORK_PARAMETERS structure (24 Bytes), leaving a total of 2022 Bytes for PARS.  However, my experiments have shown that if its size is 704 Bytes, then shortly after I call fnSaveNewPars(SAVE_NEW_PARAMETERS), my program hangs completely -- so completely that I have to unpower the DEMO board and restart the IDE to again communicate with the board, even to erase its memory.  If its size is reduced to 272 Bytes, the program runs just fine.  (I find the size by disassembling application.c and then looking at the Section Size associated with _cParameters.  I have confirmed by setting breakpoints that this is the same size that's sent to fnSetPar().)  Of course, the problem could be elsewhere.  For example, I might be overlaying fnRAM_code() with the longer parameter blocks.

As you can tell, I'm still experimenting with parameter blocks.

Best regards,
    Richard

[mark]

Hi Richard

I and others have found it impossible to debug with active watchdog so I am very surprised that you seem to be able to (perhaps it is however already disabled since any hanging should otherwise normally result in an automatic reset after 2s).

See this thread for techniques for disabling:
http://www.utasker.com/forum/index.php?topic=39.msg168#msg168

I must admit to never actually using really big parameter blocks but the theoretical values that you quote should be correct. When changing the parameter block size ensure that you increment its version number so that the code doesn't try to work with an invalid one (not a problem that would normally cause crashes though). Note also that ram copies are made of the parameters (parameters and temp_pars) on heap - if the parameters structure is increased by 1k it results in 2k more heap being required and it may be that this is failing as the size increases - check the heap allocation size (OUR_HEAP_SIZE in config.h) and use the routines fnHeapFree(), fnHeapAvailable() and fnStackFree() to monitor these. I will have a try with big parameter blocks and look at your project when you have sent it.

Regards

Mark

[Richard]

I must admit to never actually using really big parameter blocks but the theoretical values that you quote should be correct. When changing the parameter block size ensure that you increment its version number so that the code doesn't try to work with an invalid one (not a problem that would normally cause crashes though).

I did increment the version number.

Note also that ram copies are made of the parameters (parameters and temp_pars) on heap - if the parameters structure is increased by 1k it results in 2k more heap being required and it may be that this is failing as the size increases - check the heap allocation size (OUR_HEAP_SIZE in config.h) and use the routines fnHeapFree(), fnHeapAvailable() and fnStackFree() to monitor these.

The heap size in config.h is 19 KBytes.  Just before fnSaveNewPars(SAVE_NEW_PARAMETERS) is called, nHeapFree() returns 964,  fnHeapAvailable() returns 19456 and fnStackFree() returns 5767.

I will have a try with big parameter blocks and look at your project when you have sent it.

I've not sent my project, because it relies on having a 16-key keypad, 11 7-segment LEDs and six LEDs and their drivers attached.  If I can figure out some way to get these same type of results with your example project and version 7.0 of the IDE, I'll send them along.  Right now, though, I have to rush off to hear Sir Neville Marriner conduct the Academy of St. Martin in the Fields.
 
Best regards,
    Richard

[mark]

Hi Richard

The heap size in config.h is 19 KBytes.  Just before fnSaveNewPars(SAVE_NEW_PARAMETERS) is called, nHeapFree() returns 964,  fnHeapAvailable() returns 19456 and fnStackFree() returns 5767.

Is this before increasing the parameter structure? You seem to have adequate reserve on the stack. Heap free is 964 bytes so this would allow a futher increase of 482 bytes (2 copies) before it fails. Does this correspond with your results?
If you increase OUR_HEAP_SIZE by 2 x extra struct size you shoudl be OK (as I said, the stack seems to be adequate, however it is best to test after running for a short time since it measures the largest stack use. Also some TCP stuff only takes heap when it is used for the first time - like Telnet - so the heap value has be be monitored after all services have been active at least once).

Regards and enjoy your evening!!

Regards

Mark

[Richard]

Hi Richard

The heap size in config.h is 19 KBytes.  Just before fnSaveNewPars(SAVE_NEW_PARAMETERS) is called, fnHeapFree() returns 964,  fnHeapAvailable() returns 19456 and fnStackFree() returns 5767.

Is this before increasing the parameter structure?

This was after increasing the size of the structure; i.e., PARS has size 704.

In another experiment, I uploaded your web pages to be sure that a lot of things have run, and then I accessed the Administration page.  The Memory Utilisation table there shows
    HEAP Free 0x03c4 from 0x4c00
    STACK Unused 0x10a3

I then perform the keypad sequence that results in accessing the Heap and Stack functions. fnHeapFree() returns 964,  fnHeapAvailable() returns 19456 and fnStackFree() returns 4259.  The first two are as before, and the hex values for the free heap and unused stack shown on the web page agree with the decimal values returned by the functions.

So, it looks like there's enough room.  I'll work on this more tomorrow.

Regards and enjoy your evening!!

Thank you.  The concert was fabulous!

Best regards,
    Richard

[Richard]

Hi, Mark.
Let me report some intermediate results.

(1) Stability of my experiments has been vastly improved by commenting out SUPPORT_LOW_POWER. This is something I won't need, since mains power is always available to my project.  I would occasionally find my program sitting at fnDoLowPower() and be unable to get it to go further.  At least that doesn't happen anymore.

(2) Although I haven't yet figured out how to find the maximum extent of the program code, I found that erasing from the beginning of memory to 17FFF and then trying to load my program would fail.  To check, I erased the whole memory and successfully loaded the program.  I then erased from 18000 to the end of memory and found that the program would not verify.  This gives some powerful hints as to what may be happening.

(3) I then changed uFILE_START from 0x18000 to 0x20000.  This caused subsidiary problems, mainly based on built in assumptions that the file system was 80*1024 Bytes long.

To fix this, I defined a manifest constant FLASH_SIZE to be (0x40000) -- the right value for the 5223X family.  It should be somewhere in M5223X.h, but I didn't find it -- and then defining  FILE_SYSTEM_SIZE to be (FLASH_SIZE-uFILE_START).  And then I changed the various tests that checked FILE_SYSTEM_SIZE against (160*1024) to test against a constant I called FILE_SYSTEM_ADEQUATE, which I defined to be 0x20000.  I hope that this is indeed large enough!

I still have to confirm that I have managed to write the parameters to FLASH, but I wanted to let you know the outcome of the current tests.  And I'd like to know
  (a) Is there some easy way to discover, within CodeWarrior, the highest address used by the program?
  (b) Is there a reason for starting the file system at 0x18000, which corresponds to limiting the maximum program size to something smaller than that supported by the free CodeWarrior IDE?
  (c) Is 0x20000 (=128K) large enough for whatever was being required to be 160K in your original code?

Best regards,
    Richard

[mark]

Hi Richard

>>Stability of my experiments has been vastly improved by commenting out SUPPORT_LOW_POWER.
I use this for most of my work and haven't noticed any problems with it. However the advantages of it together with the M5223X with active PHY is limited since the PHY uses most power - savings are only a fraction of total consumption. I would be interested if someone else has had problems with it.
Note that it is normal that the code will be in the fnDoLowPower() call most of the time since it will sleep there until an interrupt wakes it. If you pauue the debugger it will invariably be at this code position.

>>Although I haven't yet figured out how to find the maximum extent of the program code
The demo project is set to start the parameter blocks (counted as part of file system size) at the 96k mark. If the code grows above this size it is correct that the file system must be re-dimensioned.
The best way to check this is in the map file (in \Applications\uTaskerV1.3\CodeWarrior_M5223X\uTaskerV1.3\bin in the demo project)

# Memory map:
  v_addr   p_addr   size     name
  00000800 00000800 0001596E .flash     flash
  20000000 20000000 00000400            vectorram
  20000400 0001616E 00000060 .data      sram
  20000460 20000460 000002E0 .bss       sram
  40000000 40000000 00000000 .ipsbar    ipsbar

Here it is seen that the FLASH content is extending to the 86.3k mark (example form boot loader compatible code, which starts at 0x800).

It is possible to change the linker script (eg. M52235EVB_FLASH.lcf) to reflect the length of FLASH which can be used for code (respecting your file system set up). This would then generate an error if the code grows into that space.


>>FILE_SYSTEM_SIZE against (160*1024) to test
This is removing the POST support when the file system size is too small to accept a firmware upload. It is not very important because no one would probably try to use it on a chip which was too small (I think...)

>>FLASH_SIZE to be (0x40000) -- the right value for the 5223X family
There are several in the family which have only 128k.
The following defines are presently in hw_app_m5223x.h

#define SIZE_OF_FLASH              (256*1024)
#define SIZE_OF_RAM                (32*1024)

but I may move them to the processor header file, made conditional on the exact chip type. When they are in app_hw_m5223x.h they can however easily be changed to suit if for example a M52230 is used.
The next SP will include M52210/M5222X settings too.


>>Is there a reason for starting the file system at 0x18000
The main reason is because the demo only uses 32k of the file system for web pages. This allows the set up to run in the M52230 (and others without so much FLASH) without any changes. However in the meantime the demo sets 160k file system as standard so that it works with the boot loader - this needs to be reduces to 32k when using a smaller chip otherwise the file system will cause an exception when searching through the complete memory for files (eg. FTP DIR).

>>Is 0x20000 (=128K) large enough for whatever was being required to be 160K in your original code?
As explained above, this is for the boot loader demo. It will also work with less, after configuring, but - if you are not usng it - it doesn't need to be respected. When using smaller types (like M52230) the boot loader can still be supported in external SPI FLASH. [Presently the samller chips together with SPI FLASH is proving to be the favourite method].

Regards

Mark

[Richard]

Hi, Mark.

Note that it is normal that the code will be in the fnDoLowPower() call most of the time since it will sleep there until an interrupt wakes it. If you pauue the debugger it will invariably be at this code position.

Whenever I've found it there, I've not been clever enough to get the debugger to restart the program.  This brings up another question: is there some way to get the debugger to clean up its stack and reset the microprocessor?  So far, I've had to reload the program to get this to happen.  There has to be a better way!

The best way to check this is in the map file (in \Applications\uTaskerV1.3\CodeWarrior_M5223X\uTaskerV1.3\bin in the demo project)

# Memory map:
  v_addr   p_addr   size     name
  00000800 00000800 0001596E .flash     flash
...
Here it is seen that the FLASH content is extending to the 86.3k mark (example form boot loader compatible code, which starts at 0x800).

I've searched quite a bit but cannot find documentation for the coldfire linker that explains what v_addr and p_addr are.  Is there a publicly available linker manual somewhere that I've failed to find?

It is possible to change the linker script (eg. M52235EVB_FLASH.lcf) to reflect the length of FLASH which can be used for code (respecting your file system set up). This would then generate an error if the code grows into that space.

Lack of a linker manual makes it a little tricky to alter the script or even interpret it in its current form.  However, after looking at it, it seems likely that changing the line in the MEMORY section to read
    flash2  (RX)   : ORIGIN = 0x00000420, LENGTH = 0x001FBE0
should limit the program to be below the area given over to the parameter blocks and file system (since 0x420+0x1FBE0 = 0x20000).

>>FLASH_SIZE to be (0x40000) -- the right value for the 5223X family
There are several in the family which have only 128k.

I had forgotten that.  I decided a while ago that they are too small for what I intend to do -- perhaps incorrectly, judging from your comment below -- so I put them out of my mind.

When using smaller types (like M52230) the boot loader can still be supported in external SPI FLASH. [Presently the samller chips together with SPI FLASH is proving to be the favourite method].

Is this only for the boot loader, or is it in general?  And is there some obvious reason (e.g., cost, availability)?

Many thanks,
    Richard

[mark]

Richard

is there some way to get the debugger to clean up its stack and reset the microprocessor?

Unfortunately I have also not been clever enough to find another way - I also disconnect and reload each time I need to start the program again.

a publicly available linker manual

There is a manual called "CodeWarrior Build Tools Reference Coldfire Architectures Edition" which is available somewhere on the Freescale web site. However I didn't find any reference to these in it. I don't know what they actually mean but I believe that the v_addr is the important one when the code is running - it gives the starting address of the segment. I expect that p_addr is the position of the code/variables in FLASH before copied to the run-time location. For this project (with no code running in RAM) only the initialised variables are there. Possibly the complete FLASH content is an addition of FLASH + sram.data (this could be checked in the debugger by looking for the end of code/initialised variables)

it seems likely that changing the line in the MEMORY section to read

Yes, the change is correct, but there is (unfortunately) a second setting:
___FLASH_SIZE   = 0x00040000; which probably needs to be reduced too.

I decided a while ago that they are too small for what I intend to do

This may be the case, depending on the actual project. The new M52232 and M52236 (specified to 50MHz rather than 60MHz) are however rather cheaper than the original ones and so there could be some potential to save costs in projects with high quantities.

Is this only for the boot loader, or is it in general?

This is related to the previous answer. The boot loader in SPI FLASH can generally be used, irrespective of the available internal FLASH size. Generally internal FLASH is more expensive that external FLASH. If the code can fit within 128k but web pages and firmware upload is also specified, it is probably cheaper to put these in SPI FLASH (say 512k) rather than use the larger Coldfire for it. Also from a code security perspective there is no disadvantage - using encryped upload no code is ever stored in the SPI FLASH in readable form so can be fully protected. Web pages don't need to be protected since the idea is to display them anyway..!
The new Coldfire parts (especially M52232) are rather cheaper than the original ones and so there are a lot of people looking at making the move, which makes the SPI FLASH alternative become more attractive in the process.

See parametrics at:
http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MCF5223X&nodeId=0162468rH3YTLC00M95448&tab=Buy_Parametric_Tab&fromSearch=false

Eg.
M52233 1000+ = $9.48
M52232 1000+ = $5.43

Regards

Mark