compiling in 20 minutes instead of 4 hours, making webots more packageable and easy to compile

[Kreijstal]

tired of waiting 8 hours to compile webots? come in and help, or discuss..

Continuation of PR #6749 but not complete yet, it seems webots compiles fine on windows and linux but the projects are not being compiled yet. I modify some things to make it easier to package.. your mileage may vary.
run this on msys2
pacboy -S {qt6,assimp,openvr,ois,glm,stb,popt,dlfcn,cc,cmake,pkgconf,sccache,openal}:p --needed

compiled tested with archlinux, and windows 11..

to configure build and run
you only need to git clone without submodules, then do.
python scripts/packaging/generate_proto_list.py
cmake -G"Ninja" -B build -DCMAKE_C_COMPILER_LAUNCHER=sccache -DCMAKE_CXX_COMPILER_LAUNCHER=sccache -DCMAKE_INSTALL_PREFIX=./install_dir && cmake --build build --target install&&./install_dir/bin/webots

[Kreijstal]

what do you think @CoolSpy3 ?

[CoolSpy3]

In general, I'm against maintaining two simultaneous build systems. It just doubles the amount of work required to maintain the build system and doubles the possibility of bugs. Because of that, I think that if this PR moves forward, it should also remove the existing make build system. (This means that all aspects of the workflow need to be ported to (and tested on) cmake, including dependency installation, packaging, CI, and end-user program compilation). (And/or the relevant updates need to be made to the wiki. [This should be done after merging, but it should be kept in-mind now.]) (We could also consider only porting some things to cmake, but that might create confusion. Idk, just something to think about.)

Personally, I prefer make to cmake because I understand Makefiles better than CMakeLists [so I can make bug fixes/improvements] and I like that make doesn't abstract as much away, allowing me to make very fine-grained adjustments to the build process.

That being said, that's just my preference, so I'm fine moving to cmake if that's the direction the maintainers (@omichel?) want to take this project. (It would also probably force me to eventually learn cmake, which I should really do.)

P.S. As a note on this PR title, Webots typically takes 20-30 mins to build from scratch in CI (including dependency installs), and my potato VM can usually do it in no more than an hour or so. [Actually, it might be more. I rarely do a full rebuild. Usually, any changes I make can be rebuilt in a few min or two.] If it's taking you 4 hours to build, you might want to investigate to see if something's slowing down the build on your machine (maybe AV?), you could also try a multithreaded build if it's not being run automatically (-j<number of threads>).

[Kreijstal]

@CoolSpy3 thank you for your response

you can still use make in cmake, if you use the generator Make, cmake will produce makefiles. However if the code works, I prefer Ninja, as it is way faster, the thing why I consider this is better, is to make webots more packageable, so it can be provided as a normal package from a linux distro, think debian, ubuntu, archlinux, fedora, and currently the build system is very webots specific and doesnt play nice with packaging. (You could also just use a flatpack, but many users like to use native programs instead of jailed ones), also on cmake you can configure a buildcache like sccache or ccache. This can speed compilation a lot.

This should also make it easy to users to get started as they don't need to do so much voodoo with the paths, or run source scripts. you just need the cmake invocation.

It also compiles with clang on windows, so you could probably also build it on windows on arm

[CoolSpy3]

Regardless of whether you can create Makefiles with cmake, the bit that's ultimately maintained and committed should be the cmake files. If we start maintaining a separate set of Makefiles, they would just be overwritten whenever someone wants to regenerate them, and any changes wouldn't get ported to other generators, so from a maintainer pov, it might as well just be CMake.

AFAIK, the current build system also caches intermediary products. Not sure how this differs from ccache/sccache.

I can't really comment on how accessible the current build system is to packagers. Most of that's handled by scripts/packaging, which I haven't looked at in-depth. Nevertheless, I agree that a properly implemented CMake system could probably eliminate that segment while enabling easy extension of other package managers, which would be good!

[CoolSpy3]

Also, I took a crack at running clang-format. If you want, you can try cherry-picking 3c66fbe off of DeepBlueRobotics/webots:kreijstal-cmake, and seeing if that fixes the sources test.

[Kreijstal]

Also, I took a crack at running clang-format. If you want, you can try cherry-picking 3c66fbe off of DeepBlueRobotics/webots:kreijstal-cmake, and seeing if that fixes the sources test.

Thank you I think despite me using clang-format with the provided file, it seems clang-format is version dependendent, it did a lot of other unrelated changes...