Author Topic: memory allocation  (Read 25533 times)

Offline robo

  • Newbie
  • *
  • Posts: 40
    • View Profile
memory allocation
« on: January 10, 2009, 06:35:30 PM »
Hi,

In uTasker there is function uMalloc for memory allocation, but is there function to free this memory?

Regards,
robo 
« Last Edit: January 10, 2009, 08:42:23 PM by robo »

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: memmory allocation
« Reply #1 on: January 10, 2009, 08:44:34 PM »
Robo

No, there is no uFree() associated with uMalloc(). uMalloc() is designed for use when structures need to be dynamically assigned but then exist for as long as the system is running. In fact most memory allocations do fall in to this class in small embedded systems (allocation of driver buffers, TCP/IP stack resources etc.).

The advantage of a method to allocate memory dynamically, which will not be freed, is that it is more memory efficient - it doesn't need overhead for management and so even allocation of small blocks is without any penalty. Eg. - Calling uMalloc(4) will take 4 bytes from the heap. Calling malloc(4) may require 20 bytes or so from the heap due to management overhead in the library function, making small chunk allocation quite inefficient. Note that uMalloc() will generally align chunks to 4 bytes (#define _ALIGN_HEAP_4) since this is usually better suited to CPU operation and efficiency (see also http://www.utasker.com/forum/index.php?topic=283.0)

There is also a second related function called uMallocAlign() which can be used to force the allocated memory to be aligned to a special boundary (eg. 8 or 16 byte alignment) which is used by some drivers since some hardware requires buffers to be specially aligned - some Ethernet controllers require 16 byte alignment, for example.

When malloc() free() capabilities are required in a project this can be mixed by simply using the malloc(), alloc() etc. and free() from the C-library. This means that the uMalloc() feature is not a restriction to memory management but instead an addition. However the demo project doesn't use any library components and so only needs uMalloc().

See also the following: http://www.utasker.com/forum/index.php?topic=96.0

Regards

Mark

Offline robo

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: memory allocation
« Reply #2 on: January 10, 2009, 09:05:27 PM »
Hi Mark,

Thanks for reply now it is clear for me :)
Do you have some examples how to use malloc() and free() with LM3S6965 projects?

Regards,
robo

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: memory allocation
« Reply #3 on: January 11, 2009, 12:38:20 AM »
Hi Robo

I don't have any examples (it is not processor specific), but I would simply experiment with code that does

x = malloc(10);
free(x);
x = malloc(20);
free(x);

Exactly where the library puts the heap, and how you can control its size depends on the library and compiler. You may need to experiment and search in the map file where the heap is situation. Compare with the following examples:
http://www.utasker.com/forum/index.php?topic=96.0

Normally you can control the size in the linker script. If it is included in the variable space everything should be OK since the uTasker heap expects to be on the top of all other system variables.

Regards

Mark

Offline robo

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: memory allocation
« Reply #4 on: January 11, 2009, 02:40:31 PM »
Hi Mark,

Ok, I know how to use malloc and free functions in C but I have problem with build project when I want to use it.
I'm working with gcc compiler and LM3S6965 and when I'm building project i have one error:
"1>c:/program files/codesourcery/sourcery g++ lite/bin/../lib/gcc/arm-none-eabi/4.2.3/../../../../arm-none-eabi/lib/thumb2\libc.a(lib_a-sbrkr.o): In function `_sbrk_r':
1>sbrkr.c:(.text+0xc): undefined reference to `_sbrk'
1>collect2: ld returned 1 exit status
1>cs-make: *** [uTaskerV1.3.elf] Error 1"

Do you know what can be wrong?

I have also tried biuld project in Rowley and there is no problem but after programming uP hangs in places where malloc function is in use...

regards,
robo
« Last Edit: January 11, 2009, 02:44:33 PM by robo »

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: memory allocation
« Reply #5 on: January 11, 2009, 03:45:01 PM »
Hi Robo

I also have the same linker error when compiling with CodeSourcery GCC. However there are no errors when doing the same with Rowley GCC.
This doesn't seem specific to the Luminary project but seems to be present when working with newer CodeSourcery GCC (I read also that the Yagarto chain also doesn't have the problem).

In fact if you search the web for sbrkr.c you will find that this is a typical difficulty. It seems to be that this GCC version's libraries expect the OS to deliver some 'stubs', or at least a few variables, which can sometimes even by dummy ones.


http://en.mikrocontroller.net/topic/139040
http://www.luminarymicro.com/component/option,com_joomlaboard/Itemid,92/func,view/id,886/catid,8/
https://support.codesourcery.com/GNUToolchain/kbentry16    <- if you are registered with CS you may get a solution here
http://www.st.com/mcu/forums-cat-7542-23.html
http://sourceware.org/newlib/libc.html#SEC196

whereby I think the most promising tip is http://sourceware.org/newlib/libc.html#Stubs


However this looks as though it might involve a fair bit of work to understand the internal workings and get a suitable solution.
Note that Rowley adds their own library function (not GCC based) and this seems thus complete. Perhaps the Yagarto chain adds the stubs on some startup code and these could be simply copied(?)

If you simply add the following to the code it will then link:
unsigned char *_sbrk(int incr) {return 0;}

But (presumably) the _sbrk() function needs to be filled out correctly for it to really operate correctly.
Again, a search for _sbrk() on the Internet gives a lot of ideas for doing this. Perhaps also one which can be simply integrated?

Tell me if you solve the problem - I have put it on the list of 'things to do' but probably won't be a able to actively try for the next few days or so...

Regards

Mark


Offline robo

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: memory allocation
« Reply #6 on: January 11, 2009, 03:57:36 PM »
Hi Mark,

Before I started with uTasker I had same problem, in Eclipse, and it disappear when i add syscalls.c file, but now I don't know where I must add this and how?

//syscalla.c
#include <stdio.h>
#include <sys/stat.h>

char str[100];
char *cp = str;

void writechar(char ch);


int _close(int file) {
    return -1;
}

int _exit(int x) {
    while (1);
}

int _fstat(int file, struct stat *st) {
    st->st_mode = S_IFCHR;
    return 0;
}

int _lseek(int file, int ptr, int dir) {
    return 0;
}
int isatty(int file) {
    return 1;
}

int _read(int file, char *ptr, int len) {
    return 0;
}

extern int  __HEAP_START;

caddr_t _sbrk ( int incr )
{
  static unsigned char *heap = NULL;
  unsigned char *prev_heap;

  if (heap == NULL) {
    heap = (unsigned char *)&__HEAP_START;
  }
  prev_heap = heap;

  heap += incr;

  return (caddr_t) prev_heap;
}


int _getpid() {
    return 1;
}

#include <errno.h>
#undef errno
extern int errno;
int _kill(int pid, int sig) {
    errno=EINVAL;
    return (-1);
}

void writechar(char ch) {
    *cp++ = ch;
    if (cp == (str + 100))
        cp = str;
}

int _write(int file, char *ptr, int len) {
    int todo;
    for (todo = 0; todo < len; todo++) {
        writechar(*ptr++);
    }
    return len;
}

regards,
robo

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: memory allocation
« Reply #7 on: January 11, 2009, 05:15:15 PM »
Hi Robo

If you have the file, add it to the make file with 2 additional entries:

In the OBJ list add:
       Build/syscalla.o \

Then a new compile dependency:
Build/syscalla.o: ../syscalla.c $(DEPENDS)
      arm-none-eabi-gcc $(C_FLAGS) -I../../uTaskerV1.3 -D _GNU -D _LM3SXXXX -g -c -Os ../syscalla.c -o Build/syscalla.o


This assumes that syscalla.c is in the application source directory.
If it compiles the original linking error should be removed. It may link - but it may find new missing things, so the only way to know is to try it.

Afterwards it is still necessary to check that the library's heap is in the correct place in the memory map (see map file) and that its heap space size is suitable (see which define it is using to do this and adjust of necessary).

Regards

Mark

Offline robo

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: memory allocation
« Reply #8 on: January 24, 2009, 01:48:02 PM »
Hi Mark,

When I only add syscalls.c doesn't fix the problem. I think that it need integrate with uTasker system...
I tried also to biuld project with malloc in Rowley and it compile but when program is running it hangs in place where malloc is used. How to solve this problem?

regards,
robo

Offline Lukee

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: memory allocation
« Reply #9 on: January 24, 2009, 01:58:14 PM »
Hi Mark,

I tried the same what robo and I have the same problem.

I think possobility to use malloc/free function is very important and it is additional utility.

Best regards
Lukee

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: memory allocation
« Reply #10 on: January 24, 2009, 11:36:10 PM »
Hi Robo / Lukee

I may have a solution for GCC (I will look into the Rowley case later - although it uses GCC they have added their own libraries and I could imaging that the position of heap memory needs (somehow) adjusting). I also suggest checking the following define in uTaskerLM3SXXXX.ld, which is the linker script file in the project - shared by GCC standalone build and Rowley build:

  __HEAPSIZE__ = 0;


This is the setting as the file is delivered. This means that the heap management has in fact no RAM to use. If you haven't changed this, please give it a suitable value - how much memory you would like to be reserved for the GCC heap (that is the one which malloc() and free() work with). This heap is a different one to the uTasker heap, which is used by uMalloc() and uMallocAlign().
Perhaps this already solves the Rowley case (?). Please report back so that I can then start with an investigation if necessary.


Now back to the GCC library case. I am surprised that this problem is so well documented but a clear answer and guide as to what to do about it is so elusive!! But, starting with the syscalla.c example earlier on in the thread I did some tests and have a possible solution that seems to work in my tests.

1) Set the required heap size as explained above in uTaskerLM3SXXXX.ld
Eg.
  __HEAPSIZE__ = 1024;
- this space is then reserved on the top of variables. This is compatible with the uTasker uMalloc since its heap is subsequently placed on top of this.

2) Add #include <stdlib.h> to config.h.
This will allow code anywhere in the project (all files should simply include #include "config.h") to use malloc(), free() etc.

3) Add the following routine. I added it to application.c so that I didn't have to add another file to the project. This is adequate for tests and it can be placed somewhere else later if needed.


#if defined _GNU
extern int  __heap_start__;

extern void *_sbrk(int incr)
{
    static unsigned char *heap = NULL;
    unsigned char *prev_heap;

    if (heap == NULL) {
        heap = (unsigned char *)&__heap_start__;
    }
    prev_heap = heap;

    heap += incr;

    return (void *)prev_heap;
}
#endif


This is the missing _sbrk() stub which seems suitable for the project's environment.
Note that it is conditional on _GNU. This is also used for Rowley so it may cause a conflict there (I haven't tried yet) (?) - I tested with standalone CodeSourcery g++ lite.

I am not sure exactly how it works but I see that _sbrk() is called the first time that malloc() is called, with a value of incr of 0x20 (when I simply request 1 byte from heap). It sets its local pointer *heap to the start of heap (address as seen in the map file) and increments by 0x20. It is then immediately called again with an incr value which seems to align *heap to a 4k RAM boundary. Subsequent malloc() and free() calls didn't result in further _sbrk() calls but all seemed to operate correctly.

The only way to be absolutely sure about what is happening is to study the GCC source code (which I haven't done yet).

Can you give this a try in your projects to see whether it is basically workable for yourselves?

Thanks

regards

Mark


Offline robo

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: memory allocation
« Reply #11 on: January 25, 2009, 01:03:30 PM »
Hi Mark,

Your solution working very fine :) I made tests also on real hardware.
I made tests with various allocation size and it works! But always _HEAPSIZE_ should be 20 greater than maximum allocated memory
for example:
when we use:
malloc(1024);
_HEAPSIZE_ = 1044;
and then is no problem.

Thanks for your help!!!

Regards,
robo

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: memory allocation
« Reply #12 on: January 25, 2009, 01:18:16 PM »
Hi Robo

I found the following thread a Luminary micro. It discusses the malloc(), _sbrk() use but is not completely compatible with the uTasker requirement:
http://www.luminarymicro.com/component/option,com_joomlaboard/Itemid,92/func,view/id,3975/catid,6/limit,6/limitstart,0/

I posted some results there to maybe get feedback to confirm the solution.

In the meantime I would also like to modify the _sbrk() code slightly based on new knowledge.

Code: [Select]
extern int  __heap_start__;
extern int  __heap_end__;

extern void *_sbrk(int incr)
{
    static unsigned char *heap = NULL;
    unsigned char *prev_heap;

    if (heap == NULL) {
        heap = (unsigned char *)&__heap_start__;
    }
    prev_heap = heap;

    if ((heap + incr) >= (unsigned char *)&__heap_end__) {
        return 0;
    }
    heap += incr;
    return (void *)prev_heap;
}

Here a check of the top of heap has been added. I understand that _sbrk() is called in 2 situations. Firstly to get the start address of heap, and secondly to increase the heap size if required. This code introduces a check of the growth of the heap so that it stays within its limits.

One surprising test result was that it seems as though malloc() can fail if the heap size allocated is less that 4k in size - then it will never be able to start up correctly. In fact the value is 4k + 1 byte to be exact!
Therefore also ensure the following minimum define for __HEAPSIZE__ = 4097;

The next problem is that the _sbrk() operation tries to increase heap size (once the old limit has been passed) in increments of 4k. In the worst case (and depending on the exact physically address of the heap) it may request another 4k to work with even though only a couple of bytes are really required. To be on the safe side a margin of an additional 4k may be advisable to avoid unexpected failure die to small changes in memory layout and additional requests of only a few bytes!!!

Regards

Mark

Offline mark

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3234
    • View Profile
    • uTasker
Re: memory allocation
« Reply #13 on: January 25, 2009, 01:41:56 PM »
Hi

After some more tests I have added some more notes at the Luminary Micro forum:
http://www.luminarymicro.com/component/option,com_joomlaboard/Itemid,92/func,view/catid,6/id,3976/#3976

I now recommend at least 8k (plus some overhead) for GCC heap size to ensure around 4k minimum heap use.
I am not sure where the 4k block comes from but it may be a parameter when the library is compiled.

However to be sure, be a little generous!

Regards

Mark


Offline robo

  • Newbie
  • *
  • Posts: 40
    • View Profile
Re: memory allocation
« Reply #14 on: January 25, 2009, 01:56:10 PM »
Hi Mark,

Thanks one more! :)

Regards,
robo