watch memory allocation c++ libraries SW4STM32

Development environment specific, Arduino, Eclipse, VS2013, Em::Blocks etc
Post Reply
sheepdoll
Posts: 32
Joined: Fri Dec 20, 2019 6:47 pm

watch memory allocation c++ libraries SW4STM32

Post by sheepdoll »

So last week I got my assembled STM32F407 pipe organ interface boards. After cutting a few traces and adding half a dozen blue wires they work.

I had been performing some of my initial testing with an STM429 Nucleo. I went with the F407 for the 100 pin lqfp. The old board used an 83848 PHY Which i had tested with the F407-discovery.

The code is a mix of Arduino C++ libraries and some proprietary C++ code that uses templates. The program works, but started exhibiting some strange errors. In looking at the memory addresses, the stack is crashing into the heap.

I am using SW4STM and gdb to compile and debug. I know enough C++ to be dangerous. (doubt I could create a namespace and template from scratch, but I can use them as black boxes.)

The program is failing in the New operator. The pointer that new is allocating to is in a large array of 256 objects. The last 25 or so pointers are being trashed by the stack.

I noticed that there is a menu option called profile tools with a profile memory option. This seems to require the installation of a tool called valgrind. I looked that up, but most of the online info seems dated patch to run on OSX Mavericks or irrelevant (I am cross compiling and do not want to profile my system memory.)

There really does not seem to be anyway in the GDB to see exactly where the memory is being allocated. I tried increasing the the values in the linker script. But these only seem to affect the compile time allocations. C++ likes to hide things, so it is not always easy to set a watch and see what a value is.

I also got out the reference manual and looked up the section on CCM. It does not look like anything is being linked into this 64K. Perhaps I should put some of the static objects into it. I think there is only 2K or so allocated at compile time in the .BSS section But things like the Ethernet buffers use DMA and If I read things one way DMA can not access CCM. Not exactly sure how one might go about putting some of the dynamic allocated class data (linked lists) into the the CCM. There seems to be 3 different ways to store things in CCM according to the comments in the linker script. I think there is also an attribute for placing a value into a given memory space. Anyone have practical experience with using CCM?

Eventually I need to port the Ethernet boot loader from the AVR32UC3. This deals with protected memory as the code is proprietary and is also technically IoT as the router can be wireless. So parts of the Ethernet packets are encrypted. In the AVR32 one can read the fuses and turn on a LED if the protection is enabled. There is a lot to read and absorb in the app notes. Anyone have experience with Ethernet bootloaders? I think most of the work here is with DFU. I did not connect anything to the usb pins (I need as many GPIO pins as possible well about 8 plus power and ground on a 26 pin ribbon cable.)

I also got mixed up and connected USART6 instead of USART1 so not I do not think the serial bootstrap would work as is. I have been using the ST-Link through the SWD to do the debugging.

Probably the best solution in the long run is to use a 100 pin STM429 with the additional memory. In the meantime, I am looking for ways to get the F407 working. There are 256 possible board addresses, so I can not make the allocation smaller as this works as a lookup table into the array of classes (which are templates to linked lists) It is these objects I need to watch in the debugger.
stas2z
Posts: 131
Joined: Mon Feb 24, 2020 8:17 pm
Answers: 8

Re: watch memory allocation c++ libraries SW4STM32

Post by stas2z »

sheepdoll wrote: Mon May 18, 2020 7:50 pm I also got out the reference manual and looked up the section on CCM. It does not look like anything is being linked into this 64K. Perhaps I should put some of the static objects into it. I think there is only 2K or so allocated at compile time in the .BSS section But things like the Ethernet buffers use DMA and If I read things one way DMA can not access CCM. Not exactly sure how one might go about putting some of the dynamic allocated class data (linked lists) into the the CCM. There seems to be 3 different ways to store things in CCM according to the comments in the linker script. I think there is also an attribute for placing a value into a given memory space. Anyone have practical experience with using CCM?
I had some practical experience with CCM
basically, it's a separate segment of ram and yes, you can't use DMA with it.
i think your idea about putting some static data to CCM is optimal
just select some huge static data like precalced tables, samples probably, anything static and fat and tell gcc to store it to ccm

Code: Select all

uint8_t hugeArray [32768] __attribute__((section(".ccmram")));
.ccmram section have to be defined in the linker script for sure
for example, if you are using RTOS it's a good idea to put RTOS heap there
sheepdoll
Posts: 32
Joined: Fri Dec 20, 2019 6:47 pm

Re: watch memory allocation c++ libraries SW4STM32

Post by sheepdoll »

Yeah that is what I am experimenting with. I found a reference to a 'post link' option called --format=sysv which gives a more verbose map of the memory usage.

found an interesting description on the OpenSTM32 web page. https://www.openstm32.org/Using%2BCCM%2BMemory I think when combined with this post on stack overflow https://stackoverflow.com/questions/588 ... w-operator I may have a solution to how to locate a class in .ccmram

Still a bit fuzzy about which method to construct the class data. Looks like there are many options to choose from.

__attribute__((section(".ccmram"))); does allocate the class storage in the memory map. but according to everything I have read it is uninitialized. This may not be a problem if placement-new is used to instill the class data constructor.

In the above link there is a reference to a memory section called .ccmdata which is not in the default linker script. Searching for this recursivlly points back to the self same link. OpenSTM32 has low traffic volume, with most questions remaining unaswered, I think it lacks a sense of community. My guess is one would use the section attribute to locate the data inside the .ccmdata and the startup code does the copying. Not quite clear on how it detects where in the flash the data to copy is?

If course I could put all the static init variables aka 'globals' into CCMRAM but that seems overkill. Placement-new might be a better options becouse these are arrays of pointers which get updated a lot.

The other thing is to put the stack into CCMRAM where it is unlikley to overflow into the heap allocations.

If anything there are too many choices. Surprisingly all of the variations I have tried have compiled without error. Have not tried running any of it yet.
stevestrong
Posts: 502
Joined: Fri Dec 27, 2019 4:53 pm
Answers: 8
Location: Munich, Germany
Contact:

Re: watch memory allocation c++ libraries SW4STM32

Post by stevestrong »

An example how to relocate the stack in CCMRAM: https://github.com/rogerclarkmelbourne/ ... nc#L41-L43
Eventually combined with startup code to copy rodata into ccmram: https://github.com/rogerclarkmelbourne/ ... #L172-L187
Example to locate a variable in CCMRAM: https://github.com/rogerclarkmelbourne/ ... ore.c#L168
ag123
Posts: 1655
Joined: Thu Dec 19, 2019 5:30 am
Answers: 24

Re: watch memory allocation c++ libraries SW4STM32

Post by ag123 »

there is a piece of codes i found in greiman's SdFat
freestack
https://github.com/greiman/SdFat/blob/m ... reeStack.h
pretty useful on stm32f103 20k sram
but it can't account for rtos etc, only plain simple stacks and heap.
in libmaple core and arduino sketches it works, i'm not sure about official core
sheepdoll
Posts: 32
Joined: Fri Dec 20, 2019 6:47 pm

Re: watch memory allocation c++ libraries SW4STM32

Post by sheepdoll »

So putting a simple array into CCMRAM works

Putting the stack into CCMRAM fails at the first call to new. It seems from the mangled disassembly of the new() implementation malloc is returning an exception. Not sure where SW4STM32 keeps it's library source for things like malloc.

Next up will be to try a placement new on a C class.

The FreeStack code looks interesting. Not sure how one would use it. It is obvious where the code is failing in new() which is trying to allocate an array of 256 objects all of which are class pointers. By the time the code hits the breakpoint the collision has already happened. If one does a stack trace the failure once again is related to the new() operator which is silently calling malloc.
stevestrong
Posts: 502
Joined: Fri Dec 27, 2019 4:53 pm
Answers: 8
Location: Munich, Germany
Contact:

Re: watch memory allocation c++ libraries SW4STM32

Post by stevestrong »

Review your compiler and linker options.
Maybe

Code: Select all

-nostdlib -fno-rtti -fno-exceptions -fno-use-cxa-atexit -fno-threadsafe-statics
and

Code: Select all

--specs=nano.specs --specs=nosys.specs
helps.
ag123
Posts: 1655
Joined: Thu Dec 19, 2019 5:30 am
Answers: 24

Re: watch memory allocation c++ libraries SW4STM32

Post by ag123 »

on stm32, i've yet to try 'new' the notion is that literally malloc() isn't there, it is just that 20k (or some other) amount of sram.
so so far i've limited myself to straight flat global variables, pre-allocated arrays so they sit on the heap and otherwise local variables, they sit on the stack (build downwards). using this 'highly simplified' model, freestack() works!

i've toyed with the notion of using malloc() and new, i think to use them u'd need to make a working setup of malloc() or rather _sbrk()?
that malloc probably is needed to manage ram say in ccm ram

there are some rather interesting reading materials, attempts by others who made the attempt
https://arobenko.gitbooks.io/bare_metal ... n_mem.html
https://stackoverflow.com/questions/531 ... bare-metal
https://embeddedartistry.com/blog/2019/ ... heap_limit
https://gcc-renesas.com/wiki/index.php? ... he_heap%3F
https://tty.uchuujin.de/2016/04/stm32-f ... atch-heap/

_sbrk() is quite a 'little known' if mysterious piece to play with especially if one wants to play with 'complicated' memory allocation ccm ram, main sram & all. i've yet to make that attempt ;)
sheepdoll
Posts: 32
Joined: Fri Dec 20, 2019 6:47 pm

Re: watch memory allocation c++ libraries SW4STM32

Post by sheepdoll »

I got it working. _sbrk was indeed the key. Put a breakpoint there and it was exactly what I wanted. I could see the end of heap pointer.

To use placement new, one also has to '#include <new>" which is a bit of a strange syntax. I guess <new> maps to new.h

That said it was not a clean solution. If the class contains an array of subclass new() still calls malloc(). So the base class goes into ccmram, but the array of subclass still goes onto the heap.

This is also true when one defines the class as an automatic variable. In that case the startup code calls new() before the HAL_INIT() code is called in main. I was using the ADAFRUIT tft library, which starts the TFT when the class is instantiated. This failed on HAL_Delay() as the timers were not on. The work around is to define a char array with the sizeof on the class. Then after the startup code and beo

_sbrk was also where the code was failing when I moved the stack to ccmram. It checks to see if the stack is less than the start of the heap plus the break. Since the ccmram ends at 0x1001FFF and the heap starts at 0x2002000 it always catches the exeption. The work around would be to change _sbrk which is in the file syscalls.c

The caveat is that _sbrk can be called at any time, even as part of the start code. So it would be a bit difficult to tell it which memory space the block goes into. The other issue is that the startup code runs with the stack at the base level (usually top of ram) So new malloc happily allocates the structures. Later when functions are called, the stack overwrites things as it has no way of knowing that it passed the end of the heap.

Next up is to write the ethernet bootloader. Probably start a new thread for that...
Post Reply

Return to “IDE's”