Precompiled HAL/LL files possible?

Post here all questions related to STM32 core if you can't find a relevant section!
ag123
Posts: 1655
Joined: Thu Dec 19, 2019 5:30 am
Answers: 24

Re: Precompiled HAL/LL files possible?

Post by ag123 »

i think in about all the Arduino IDE , arduino cli and various tools e.g. vs code etc, there is a way to specify the *build folder* rather than using arbitary *temp folders*. hence, you could get 'incremental build' simply finding out how to set a fixed build folder rather than using temp folders.
i'd prefer build folders to be part of the project folders so that compiled objects don't get mixed across sketches

that's how i used my makefiles. i'm using eclipse IDE and initially there is a lot of pains with each different sketch / project i've to manually setup all the defines, all the include paths, all the compiler settings then all the source locations, all the library (including cmsis etc) locations.
so if i've blink1 sketch, to make blink2 sketch it wastes 1/2 a day setting those definitions up.

makefiles fixes it all, now that when i want to create a new sketch, i clone the project folders and edit settings e.g. in makefile etc.
it reduce that from 1/2 day to a few minutes.

arduino-cli is possibly more portable than makefiles between windows and linux (as it is done by design). but in windows if you use WSL (windows subsystem for linux)
https://docs.microsoft.com/en-us/window ... tall-win10
using makefiles or even docker based builds could be easy as well.

i think *docker* is a 'technology' we can explore as well, as it makes possible to bundle a whole environment for the build, and it can be pretty consistent across different platforms. the trouble is that these bundles tend to be very large files. another downside of docker is that the tight bundling discourage sharing of common tools across different cores
BennehBoy
Posts: 135
Joined: Sat Jan 04, 2020 2:38 am
Answers: 1

Re: Precompiled HAL/LL files possible?

Post by BennehBoy »

I think anything beyond installing Arduino, adding the core json URL & adding the core, is asking most of the intended audience too much.

So whilst more can be achieved coding in a different IDE, it's 'at your own risk'.

Might be worth having some 'how to get better performance' tips for the base setup in a thread somewhere however.
mrburnette
Posts: 633
Joined: Thu Dec 19, 2019 1:23 am
Answers: 7

Re: Precompiled HAL/LL files possible?

Post by mrburnette »

MGeo wrote: Thu Mar 25, 2021 8:08 am Thanks for the replies. I sounds like a smarter IDE with incremental build features could provide an improvement on a project by project basis (ie each quick and dirty test project will still drive a new folder / full rebuild).
So, with IDE 2.0 Beta, I did an STM32duino build, then I did a rebuild. See the file list below:
Compiling sketch...
"C:\\Users\\Ray\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\4.8.3-2014q1/bin/arm-none-eabi-g++" -c -g -Os -Wall -Wextra -DDEBUG_LEVEL=DEBUG_ALL -std=gnu++11 -MMD -ffunction-sections -fdata-sections -nostdlib --param max-inline-insns-single=500 -fno-rtti -fno-exceptions -fno-use-cxa-atexit -DBOARD_maple_mini -DVECT_TAB_ADDR=0x8005000 -DERROR_LED_PORT=GPIOB -DERROR_LED_PIN=1 -mcpu=cortex-m3 -DF_CPU=72000000L -DARDUINO=10607 -DARDUINO_MAPLE_MINI -DARDUINO_ARCH_STM32F1 -DMCU_STM32F103CB -DSERIAL_USB -mthumb -march=armv7-m -D__STM32F1__ -DMCU_STM32F103CB -DSERIAL_USB -mthumb -march=armv7-m -D__STM32F1__ "-IC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\system/libmaple" "-IC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\system/libmaple/include" "-IC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\system/libmaple/stm32f1/include" "-IC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\system/libmaple/usb/stm32f1" "-IC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\system/libmaple/usb/usb_lib" "-IC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\cores\\maple" "-IC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\variants\\maple_mini" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\sketch\\AnalogTest.ino.cpp" -o "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\sketch\\AnalogTest.ino.cpp.o"
Compiling libraries...
Compiling core...
Using previously compiled file: C:\Users\Ray\AppData\Local\Temp\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\core\wirish\start.S.o
Using previously compiled file: C:\Users\Ray\AppData\Local\Temp\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\core\wirish\start_c.c.o
Using previously compiled file: C:\Users\Ray\AppData\Local\Temp\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\core\wirish\syscalls.c.o
Using previously compiled file: C:\Users\Ray\AppData\Local\Temp\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\core\wirish\boards.cpp.o
Using previously compiled file: C:\Users\Ray\AppData\Local\Temp\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\core\board.cpp.o
Using previously compiled file: C:\Users\Ray\AppData\Local\Temp\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\core\wirish\boards_setup.cpp.o
Using precompiled core: C:\Users\Ray\AppData\Local\Temp\arduino-core-
cache\core_STM32_STM32F1_mapleMini_2d4a391f166aa6403f9ab47e6d350f31.a
Linking everything together...
"C:\\Users\\Ray\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\4.8.3-2014q1/bin/arm-none-eabi-g++" -Os -Wl,--gc-sections -mcpu=cortex-m3 "-TC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\variants\\maple_mini/ld/flash.ld" "-Wl,-Map,C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0/AnalogTest.ino.map" "-LC:\\Users\\Ray\\Documents\\Arduino\\hardware\\STM32\\STM32F1\\variants\\maple_mini/ld" -o "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0/AnalogTest.ino.elf" "-LC:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0" -lm -lgcc -mthumb -Wl,--cref -Wl,--check-sections -Wl,--gc-sections -Wl,--unresolved-symbols=report-all -Wl,--warn-common -Wl,--warn-section-align -Wl,--warn-unresolved-symbols -Wl,--start-group "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\sketch\\AnalogTest.ino.cpp.o" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\core\\wirish\\start.S.o" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\core\\wirish\\start_c.c.o" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\core\\wirish\\syscalls.c.o" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\core\\board.cpp.o" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\core\\wirish\\boards.cpp.o" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0\\core\\wirish\\boards_setup.cpp.o" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-core-cache\\core_STM32_STM32F1_mapleMini_2d4a391f166aa6403f9ab47e6d350f31.a" -Wl,--end-group
"C:\\Users\\Ray\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\4.8.3-2014q1/bin/arm-none-eabi-objcopy" -O binary "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0/AnalogTest.ino.elf" "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0/AnalogTest.ino.bin"
"C:\\Users\\Ray\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\4.8.3-2014q1/bin/arm-none-eabi-size" -A "C:\\Users\\Ray\\AppData\\Local\\Temp\\arduino-sketch-7C9F19D26B063DA5C2264F74A92FEDD0/AnalogTest.ino.elf"
Sketch uses 16276 bytes (14%) of program storage space. Maximum is 110592 bytes.
Global variables use 3136 bytes (18%) of dynamic memory, leaving 14272 bytes for local variables. Maximum is 17408 bytes.

--------------------------
Compilation complete.
In my experience, IDE 1.8.13 works rather similarly.
MGeo
Posts: 38
Joined: Thu Dec 19, 2019 10:29 am
Answers: 2

Re: Precompiled HAL/LL files possible?

Post by MGeo »

fpiSTM wrote: Thu Mar 25, 2021 10:22 am I don't know. Anyway some of the drawback on Windows are the network access and the antivirus...
Hmm, I'm not sure that 'just use Linux because it decides to not worry about scanning for viruses' is a durable argument. It is certainly not an option in work environments. I do have dual boot Ubuntu and am comfortable with it, but it sure seems like incremental build options, archive files, etc are worth exploring.
Last edited by MGeo on Sat Mar 27, 2021 10:40 am, edited 1 time in total.
mrburnette
Posts: 633
Joined: Thu Dec 19, 2019 1:23 am
Answers: 7

Re: Precompiled HAL/LL files possible?

Post by mrburnette »

MGeo wrote: Thu Mar 25, 2021 9:40 pm
fpiSTM wrote: Thu Mar 25, 2021 10:22 am I don't know. Anyway some of the drawback on Windows are the network access and the antivirus...
Hmm, I'm not sure that use Linux because it decides to not worry about scanning for viruses is a durable argument. It is certainly not an option in work environments. I do have dual boot Ubuntu and am comfortable with it, but it sure seems like incremental build options are worth exploring.
IF your employer manages work PC from domain policy then you are at their mercy ... after all, it is their time they are 'burning.'

If you have Admin rights, then you have 2 options:
Win10DefenderFix.JPG
Win10DefenderFix.JPG (75.68 KiB) Viewed 3284 times

As a once-upon-a-time-MCSE, I strongly suggest updating your AV, disconnecting from the Internet, doing a "full" AV-scan, fix any problems. Then, stay disconnected, invoke #1 or #2 option from above list. Compile 'til the cows come home. Undo AV changes before going back online.

IMO: Linux is the more desirable workstation environment.
ag123
Posts: 1655
Joined: Thu Dec 19, 2019 5:30 am
Answers: 24

Re: Precompiled HAL/LL files possible?

Post by ag123 »

while playing with Arduino IDE 2.0 beta 4, i found that arduino-ide 2.0 or rather arduino-cli
builds a pre-compiled .a library after all in
/tmp/arduino-core-cache/core_bdc163ff97f8266.....a

Code: Select all

> ar -t core_bdc163ff97f82660450f5a0d8e958d0b.a 
startup_stm32yyxx.S.o
dtostrf.c.o
board.c.o
hooks.c.o
itoa.c.o
pins_arduino.c.o
device.c.o
...
ok so it builds a .a lib file after all - on linux - i'm not sure if it does the same on windows
and the other thing is using /tmp directories for pre-compiled libs really isn't a good idea.
i'm not sure where can the build folder be set so that the intermediate built object files, .a libraries etc can be left in the project folder in the build subfolder.
it doesn't make sense to rebuild each time the /tmp folder is cleared

Arduino IDE 2.0 thread here
viewtopic.php?f=18&t=946
mrburnette
Posts: 633
Joined: Thu Dec 19, 2019 1:23 am
Answers: 7

Re: Precompiled HAL/LL files possible?

Post by mrburnette »

ag123 wrote: Sat Mar 27, 2021 8:53 pm ...
it doesn't make sense to rebuild each time the /tmp folder is cleared
...
I suggest it makes perfect sense as the temporary files are bound to a specific IDE sessions ... For example: 3 IDE sessions and 3 set of files are separated neatly in subdirectories beneath /tmp
Close any IDE session and the IDE does the clean-up automatically.

Between the time the IDE is closed and the source files re-opened, all manner of crap can happen; libraries updated, regressed, board files changed, etc. I have compiled by mistake with the wrong board flavor... it happens.

Opinion: PCs ... even 5 or 6 year old ones are reasonably fast and in 2021 complaining about 1st compile time on a microcontroller is somewhat silly :o

Tradition states that beer is intended to be consumed during compile + link sessions + upload, so I have seriously been thinking about slowing down my PC :D


Ray
ag123
Posts: 1655
Joined: Thu Dec 19, 2019 5:30 am
Answers: 24

Re: Precompiled HAL/LL files possible?

Post by ag123 »

well, the solution is per-project / per-sketch build folders, that's how i'm doing with makefiles currently.
the core and associated libraries (e.g. cmsis) can reside in a central location, but each project folder has is own bin or build folder
and the build outputs are placed in the build folder.this is also a common way for eclipse compile.

if the codes for the core is rather static, i think it may even be possible to use .a lib files and link those directly,
then what would work is that only the .h include files are referenced and linking uses .a lib libraries, much faster

the main thing about that 'slowness' is that the compiler has to parse the entire set of if-defs through several hundred megs? of c, cpp and .h source files to determine the appropriate permutation of soc, stm32fxxyy, board and feature sets of spi, timers, uart, i2c, dma, usb, etc used, all of them different.
if you add variants (i.e. boards) for the led (or leds) there can be n x 3 x different board quantities of the pin, active high or active low combinations.
linux and the big os that has the luxury of large storage and ram has resort to using the device tree (https://www.devicetree.org/) and resolve them at run time. interestingly pc13 seem to be a de-facto 'standard' for LED_BUILTIN or BOARD_LED_PIN following the legacy of stm32f103 blue pills.
but crystal speeds is still different between boards.

in that perspective it is actually more complicated than a desktop build, you could compare to compiling a linux kernel for x86-64 pc, that tend to target one common feature set but a very large all encompassing one. i.e. it depends on dynamic linking at run time to resolve the dependencies.
BennehBoy
Posts: 135
Joined: Sat Jan 04, 2020 2:38 am
Answers: 1

Re: Precompiled HAL/LL files possible?

Post by BennehBoy »

mrburnette wrote: Sun Mar 28, 2021 12:25 am Tradition states that beer is intended to be consumed during compile + link sessions + upload, so I have seriously been thinking about slowing down my PC :D
:lol:
mrburnette
Posts: 633
Joined: Thu Dec 19, 2019 1:23 am
Answers: 7

Re: Precompiled HAL/LL files possible?

Post by mrburnette »

Many years (9+) ago, I concocted a Windows 8.0 batch program to move compiled Arduino 1.0 files around my Win machine. I had forgotten about it, so I sat down on a Win 10 box today and created a short batch script (one could PowerShell too) to simply make a record of where the existing ELF file was located (or ELFs if multiple IDE are active,)

Just create a CMD file with the contents below and run it (depending on how your PC is configured you may need to run as Administrator.)
I call my file "WhereIsELF.cmd"

Code: Select all

REM by M.R. Burnette
REM Locate the current ELF files
ECHO OFF

ECHO %date% ON %time% >"%HOMEPATH%\Documents\ElfRhere.txt"

C:
CD \
ECHO The Arduino ELF file(s) are located here >> "%HOMEPATH%\Documents\ElfRhere.txt"

dir %temp%\*.elf /s /b >> "%HOMEPATH%\Documents\ElfRhere.txt"

CD "%HOMEPATH%\Documents"

Start Notepad.exe ElfRhere.txt
You can see that I am scanning from C:\ but you MAY want to be more restrictive here if multiple users use the box.
Example:

Code: Select all

CD %HOMEPATH%
If you have compiled something recently your output will appear similar to
Sun 03/28/2021 ON 12:49:56.31
The Arduino ELF file(s) are located here
C:\Users\Ray\AppData\Local\Temp\arduino_build_439018\ILI9341_Harichord.ino.elf
Maybe this will simplify things for some folks.

Using 2 IDE active sessions, I get something like this:
Sun 03/28/2021 ON 12:52:50.28
The Arduino ELF file(s) are located here
C:\Users\Ray\AppData\Local\Temp\arduino_build_439018\ILI9341_Harichord.ino.elf
C:\Users\Ray\AppData\Local\Temp\arduino_build_571616\BlinkNcount-Printf.ino.elf
Post Reply

Return to “General discussion”