Update toolchains to newer compilers

[quiret]

Description

This pull request updates the platform and package versions of build configurations to their latest version. The build configurations have been rigorously tested (mostly under my HAL SPI rework).

Improved generic_create_variant.py to not mess with official board variant configurations (only touch if board_build.variant value is prefixed with MARLIN_).

Benefits

Newer compilers emit better code. Newer compilers provide newer C++ language features thus more powerful software models can be implemented. Newer toolchains may come with support for even more boards.

Related Issues

#24911 (depends on this PR)

[ellensp]

I really hope you have thoroughly tested these new compilers on all boards effected, and are not just blindly upgrading them... beyond it still compiles...

[quiret]

I really hope you have thoroughly tested these new compilers on all boards effected, and are not just blindly upgrading them... beyond it still compiles...

The upgrade process is being performed slowly! You can see that I have left compatibility .ini configurations as fallback. The following boards have been tested IRL:

• BTT SKR V1.4
• MKS TinyBee V1.0
• MKS Robin E3D V1.1

Also please do not seed distrust into PlatformIO official registry packages. They have been released for a good reason and PlatformIO targets embedded devs. I will nevertheless make sure everything works ;) Progress is necessary!

[quiret]

Please watch PRs me-no-dev/ESPAsyncWebServer#1142 and luc-github/ESP3DLib#51 for merging, then change back dependencies if the official packages work.

[quiret]

I really hope you have thoroughly tested these new compilers on all boards effected, and are not just blindly upgrading them... beyond it still compiles...

@ellensp Hey I need some help testing the things that run on CI locally. Do you know how I can do that? I'd greatly appreciate it! Need to fix the issue for the MKS TinyBee which fails in an unknown Linux script...

Never mind! Got it figured out. I simply took another crash course into Marlin FW, specifically the automated build system scripts and stuff. Really interesting and worth a video! That system looks very stable so users should get an insight if they want to contribute to Marlin FW.

[quiret]

Please monitor PR espressif/arduino-esp32#7744 + check for the release of arduino-espressif32 version 2.0.7. Then remove the custom dependency from esp32.ini !

[ellensp]

I noticed that you added gnu++1z to avr env

According to this document https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html

‘gnu++17’
‘gnu++1z’
`GNU dialect of -std=c++17. This is the default for C++ code. The name ‘gnu++1z’ is deprecated.`

So perhaps you should be using gnu++17 instead ?

The document also state "This is the default" is this define even needed?

It does build slightly differently

example ramps build.
With gnu++1z
RAM: [====== ] 60.3% (used 4938 bytes from 8192 bytes)
Flash: [===== ] 51.1% (used 129668 bytes from 253952 bytes)

Without gnu++1z
RAM: [====== ] 60.3% (used 4938 bytes from 8192 bytes)
Flash: [===== ] 51.0% (used 129630 bytes from 253952 bytes)

38 bytes less flash...

[quiret]

I noticed that you added gnu++1z to avr env
(...)
The document also state "This is the default" is this define even needed?
(...)
38 bytes less flash...

Thanks for pointing this out! I am still trying to figure out the proper C++ standard targets for each platform. The C++17 standard is already pretty good, but there are really amazing features in C++20 that we should not miss out on.

It is sometimes necessary to override -std flags from dependency packages / platform options, so that is also a reason.

[quiret]

@ellensp Please note that the currently available GCC for AVR version is 7.3 on PlatformIO and you have mentioned the default -std=c++17 option that is only from GCC version 11.

https://registry.platformio.org/tools/platformio/toolchain-atmelavr/versions
https://gcc.gnu.org/projects/cxx-status.html#cxx17

Worth pointing out because GCC versions differ between platforms! Basically GCC maintenance is a huge mess due to the distributed nature of the compiler.

[quiret]

As part of my work I can fix the following issue: #24717

[EvilGremlin]

Marlin is reluctant to bump toolchain versions because it will break things, potentially for thousands people. Can you test at least dozens configurations for each MCU? And that's just hardware, not software feature conmbinations. Do you at least cople dozens most popular boards, dozen different displays and machines of every kinematic so you can debug and fix stuff you will break with toolchain updates? As long as current toolchain is satisying - we'd better not touch it.

[ellensp]

"As part of my work I can fix the following issue: #24717"

That is trivial, main issue is pins_debugging needs re written for stm32 platform, it doesn't support many MPU's properly at all.

Most Issues are with analog pins.

And this should be in a separate PR.

[quiret]

"As part of my work I can fix the following issue: #24717"

That is trivial, main issue is pins_debugging needs re written for stm32 platform, it doesn't support many MPU's properly at all.

Most Issues are with analog pins.

And this should be in a separate PR.

@ellensp I think that a proper implementation of pin debugging should be in a seperate PR. But I will adjust some things for compatibility sake.

[quiret]

Marlin is reluctant to bump toolchain versions because it will break things, potentially for thousands people. Can you test at least dozens configurations for each MCU? And that's just hardware, not software feature conmbinations. Do you at least cople dozens most popular boards, dozen different displays and machines of every kinematic so you can debug and fix stuff you will break with toolchain updates? As long as current toolchain is satisying - we'd better not touch it.

@EvilGremlin As I have said, PlatformIO packages are there for a reason. They are tried & tested by the embedded community. Thus it is safe to assume that they are ready to be deployed on real hardware. Nevertheless I will perform big tests on real hardware.

[EvilGremlin]

@quiret

PlatformIO packages are there for a reason. They are tried & tested by the embedded community. Thus it is safe to assume that they are ready to be deployed on real hardware

Not in a slightest. Marlin is obviously included in that testers community, and proven to be a benchmark for toolchains and platformio itself. There were multiople occasions where Marlin was first to hit a bug or deficiency there. Sometimes latest is better, sometimes it's much worse. You're obviously free to push forward but be ready to break stuff and introduce elusive bugs (i.e. stumbling upon compiler bug). I recommend making separate PR fot each chip family toolchain update if at all possible (yes, 6 PRs for STM)

[quiret]

(...) I recommend making separate PR fot each chip family toolchain update if at all possible (yes, 6 PRs for STM)

@EvilGremlin I agree with you on that, but on the condition that you mean separate PR for each STM chip family. This PR targets exclusively STM32F1 which is a very stable and known MCU series by ST. They are well documented. The changes I bring to Marlin related to ST are made by following their official documentation. I argue that we are on the safe side.

I have already planned to do another PR that targets STM32F4, even before you raising that concern.

[quiret]

I am putting this on hold until I have fixed some technical difficulties that arrised during testing. Please wait until I undraft this PR! In the mean time I have less technically challenging PRs that I want to put, which also drive Marlin forwards.