Update Platform, Toolchain, Compiler/GCC, C++ Versions

[TripleWhy]

Is your feature request related to a problem? Please describe.
I'm always frustrated when I'm trying to work with the code.

Describe the solution you'd like
I would like to

• use as few different platform, toolchain, compiler and c++ versions as possible,
• upgrade the C++ version to at least C++17 for all environments,
• upgrade to platform version to the latest working one,
• upgrade the compiler to the latest possible version.

Describe alternatives you've considered
Keeping everything as is is an option. Upgrading only parts of the above are options.

Additional context
There are some things I have been wanting to work on for time, but I don't want to deal ancient compilers and c++ versions. Currently we are bound to C++11 (except for esp8266, which compiles using C++17). Additionally, some of the compilers are so old, they lack support some basic flags and pragmas, such as changing optimization level for specific functions.

So far, when asking about this, I have gotten some vague answers there used to be some issues with some combinations of boards and platform version, but never any specifics. So, I would like this issue to be used to document:
a) what those issues are exactly, so that they can be tested with newer versions of things.
b) what would need to happen for this to work. For example: Wait for bug x to be fixed; drop support for older board y; upgrade only compiler but not platform for environment z.

With this documented, it should then be possible to discuss and estimate the impact and amount of work that would be, as well as what parts can or cannot be upgraded.

[netmindz]

In general I'm in favour. One thing I would say is that because of the complex relationship between the various different parts of the platform I would advise caution from creating something too bespoke as franky many of the default options are riddled with bugs so we don't want to introduce more issues.

So for example, using a different compiler than expressif expect could cause all sorts of fun.

Therefore we need to prioritise upgrading to standard configurations first, which in turn will be limited by the libraries we use as there are many breaking changes between Arduino 2 and 3 and also esp-idf 4 and 5

[blazoncek]

I think that changing toolchain is the easiest and safest option. This will include new compiler and other tools.

[netmindz]

Recent discussion mentoring updates to C++

#4484 (comment)

[blazoncek]

Recent discussion mentoring updates to C++

I think that was exactly related to this issue (or this issue is a result from that discussion). ESP32 env uses ancient gcc and working around that (by forcing newer standards via compiler options) may be prone to bugs. Upgrading toolchain may get us newer compiler (for better or worse yet to be determined).

[TripleWhy]

So for example, using a different compiler than expressif expect could cause all sorts of fun.

This is exactly the kind of vague comment I am trying to demystify here. What exactly are these sorts of fun? Without that knowledge, we can't really judge if anything improved with newer versions.

The last time I tried building wled using newer toolchains for esp32, everything seemed to work fine for what I was trying to do.

[DedeHai]

This is exactly the kind of vague comment I am trying to demystify here. What exactly are these sorts of fun? Without that knowledge, we can't really judge if anything improved with newer versions.

espressif core libraries are written and tested using a specific toolchain I am guessing. From what I have seen in those libraries they are not programmed that well or at least not as thorough as they should be. So using a different toolchain some timings might change, which can lead to unexpected behaviour or non-working peripherals. This is the "all sorts of fun". You never know what you get. But in any case, it is worth a shot but since configurations and combinateions are almost endless, it may be hard to find edge cases.

[netmindz]

So, please forgive my ignorance, but first question

Do we get any of the newer options we are looking for in the V4 vs V3 platform? If not, does the latest V4 do any better Vs our current one?

(I'm keen to ensure we first eliminate the official options before using unsupported combinations of language versions, compiler versions etc)

[TripleWhy]

Using IDF 4.4.4 upgrades gcc from 5.2.0 (July 16, 2015) to 8.4.0 (March 4, 2020) on esp32.

Does it add some of the missing flags/pragmas? IIRC, it does.
Does it upgrade from c++11 by default? No. AFAICT, at the time of writing, no upstream version does that.
Does it bring anything up to date, or even into the support window? No.
Does it upgrade to full c++17 support if used with -std=gnu++17? No, but it improves support.

[TripleWhy]

According to discord, the ESPHome guys are actively trying to get away from platformio because apparently, they stopped publishing updates for ESP platforms. Is that a path we could/should look into, too?

Will try to find what and how they do/did it, later.

On a related note... do we actually need arduino?

[DedeHai]

On a related note... do we actually need arduino?

yes, many commands and libraries depend on it.

[netmindz]

The fork I believe you are referring to is used to get Arduino Core V3 and ESP-IDF V5

Both of which contain breaking changes for the libraries that WLED is heavily dependent on.

We will however be using this platform when we come to add support for C6 and P4 chips. See MoonModules work in this area for more details

[Jason2866]

My 2 cents moving to toolchains 8.4.0 isn't a big win. There are still many bugs and c++17 support is faulty. A big step forward are the gcc13/14 toolchains used with IDF 5.x
A probably safe way to use this toolchains is to compile the Arduino libs with this toolchains.
Many errors will occur. Imho not worth the work to fix all this.
The only safe way is to move the code base to Arduino core 3.x.x or dropping Arduino completely and move to IDF. A lot of code to rewrite.

Knowing the pain of this. We had to decide for Tasmota too. In first step we optimized Arduino to our needs (mainly changed sdkconfig settings). Since Arduino 3.x has major improvements in network stack, we jumped early on this version (more changes).
Since i had the experience how to build platformio compatible frameworks (from Tasmota) and it was finally clear that Platformio would not support Arduino core 3.x.x i decided to make the pioarduino fork. esphome is using this fork for there latest versions.
Imho Platformio will not be there in business in a few years. I prepared already a version in pioarduino which is completely independent from Platformio (registry). So even in this case it will be possible to build projects (with minor changes) as it is now.