[bug] profile include() does not work with os=Linux

[paulharris]

Describe the bug

(conan 2.4.0, on linux)

I have a base profile, base-linux:

[settings]
arch=x86_64
os=Linux

I have another profile, base-clang:

include(base-linux)
[settings]
compiler=clang
compiler.libcxx=libstdc++11
compiler.version=18
compiler.cppstd=20

And I have the top level profile, reldeb-clang:

include(base-clang-18)

[settings]
build_type=RelWithDebInfo

When I build with this as the host profile, the resulting output is:

======== Input profiles ========
Profile host:
[settings]
build_type=RelWithDebInfo
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=18

Note, there is no os (and arch) listed, and the recipes complain and fail to build.

If I move the base-linux contents into base-clang, then it works.

There seems to be no documentation on the website about this behaviour (that I could see),
and seems to be a bug.
If the user can't do this, then conan should detect this problem and fail earlier with a clear error message.

How to reproduce it

No response

[franramirez688]

Hi @paulharris - Thanks for reporting this 😁

I do not get to reproduce the issue. It's working well on my side. Do you have another file named base-linux in your Conan v2 home profile? If I put an empty base-linux in the Conan profiles folder: /Users/conan/.conan2/profiles/base-linux, then the behavior is the same as described above (you can run conan profile list to check it rapidly).
To discard name collisions, let's change the file name to base-superprofile-linux to something unique or even use a relative path instead of using only the name:

base-clang

include(base-superprofile-linux)
[settings]
compiler=clang
compiler.libcxx=libstdc++11
compiler.version=18
compiler.cppstd=20

Or:

include(./base-linux)
[settings]
compiler=clang
compiler.libcxx=libstdc++11
compiler.version=18
compiler.cppstd=20

Let me know if it solved the issue.

[paulharris]

That file is the only file named "base-linux" on the filesystem.

I can confirm it is loading by putting "BREAKME" in the file, at which point it fails.
I specify a version-controlled location for my profiles.

$ conan version
version: 2.4.0
conan_path: /build/mx/devenv/bin/conan
python
  version: 3.10.12
  sys_version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
  sys_executable: /build/mx/devenv/bin/python
  is_frozen: False
  architecture: x86_64
system
  version: #24-Ubuntu SMP PREEMPT_DYNAMIC Tue May  7 14:26:31 UTC 2024
  platform: Linux-6.5.0-1023-oem-x86_64-with-glibc2.35
  system: Linux
  release: 6.5.0-1023-oem
  cpu: x86_64

$ ls
myprofs

$ ls myprofs/
base-clang-18  base-linux  release-clang-18

$ conan new cmake_exe -d name=testprofile -d version=1.0
File saved: CMakeLists.txt
File saved: conanfile.py
File saved: src/main.cpp
File saved: src/testprofile.cpp
File saved: src/testprofile.h
File saved: test_package/conanfile.py

$ cat myprofs/release-clang-18 
include(base-clang-18)

[settings]
build_type=Release

$ cat myprofs/base-clang-18 
include(./base-linux)

[settings]
# arch=x86_64
# os=Linux
compiler=clang
compiler.libcxx=libstdc++11
compiler.version=18
compiler.cppstd=20

[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang-18', 'cpp': '/usr/bin/clang++-18'}

$ cat myprofs/base-linux 
[settings]
arch=x86_64
os=Linux

$ conan build . --profile:host myprofs/release-clang-18 

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=18
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang-18', 'cpp': '/usr/bin/clang++-18'}

Profile build:
[settings]
[conf]



======== Computing dependency graph ========
Graph root
    conanfile.py (testprofile/1.0): /build/mx/testprofile/conanfile.py

======== Computing necessary packages ========

======== Installing packages ========

======== Installing packages ========

======== Finalizing install (deploy, generators) ========
conanfile.py (testprofile/1.0): Calling generate()
conanfile.py (testprofile/1.0): Generators folder: /build/mx/testprofile/build/Release/generators
ERROR: conanfile.py (testprofile/1.0): Error in generate() method, line 30
	tc.generate()
	ConanException: The 'build' profile must have a 'os' declared

$ echo "BREAKME" >> myprofs/base-linux 
$ conan build . --profile:host myprofs/release-clang-18 
ERROR: Error reading 'myprofs/release-clang-18' profile: Error reading 'base-clang-18' profile: Error reading './base-linux' profile: Invalid setting line 'BREAKME'

[franramirez688]

I just copied and pasted everything from your comment above (even the commented lines in the clang profile). Neither macOS nor Linux fails 😓

$ conan build . --profile:host profs/release-clang-18

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=18
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang-18', 'cpp': '/usr/bin/clang++-18'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux

======== Computing dependency graph ========
Graph root
    conanfile.py (testprofile/1.0): /home/develop/conan/xxxxxxx/conanfile.py

======== Computing necessary packages ========

======== Installing packages ========

======== Installing packages ========

======== Finalizing install (deploy, generators) ========
conanfile.py (testprofile/1.0): Calling generate()
conanfile.py (testprofile/1.0): Generators folder: /home/develop/conan/xxxxxxx/build/Release/generators
conanfile.py (testprofile/1.0): CMakeToolchain generated: conan_toolchain.cmake
conanfile.py (testprofile/1.0): CMakeToolchain: Preset 'conan-release' added to CMakePresets.json.
    (cmake>=3.23) cmake --preset conan-release
    (cmake<3.23) cmake <path> -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=generators/conan_toolchain.cmake  -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Release
conanfile.py (testprofile/1.0): CMakeToolchain generated: /home/develop/conan/xxxxxxx/build/Release/generators/CMakePresets.json
conanfile.py (testprofile/1.0): CMakeToolchain generated: /home/develop/conan/xxxxxxx/CMakeUserPresets.json
conanfile.py (testprofile/1.0): Generating aggregated env files
conanfile.py (testprofile/1.0): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']

======== Calling build() ========
conanfile.py (testprofile/1.0): Calling build()
conanfile.py (testprofile/1.0): Running CMake.configure()
conanfile.py (testprofile/1.0): RUN: cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="generators/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="/home/develop/conan/xxxxxxx" -DCMAKE_POLICY_DEFAULT_CMP0091="NEW" -DCMAKE_BUILD_TYPE="Release" "/home/develop/conan/xxxxxxx"
.....

I just opened a PR to simulate this too: #16454

Please, feel free to try it locally to see if it doesn't work. If the CI does not fail, let's add all the changes that could make it fail.

[paulharris]

Ok thanks, there was a problem with my test,

1. I have an empty default profile, so I needed to specify --profile:build
2. My actual profile has a second [settings] section, which is the real problem.

I'm aware that ini file formats have limitations, but having a second group results in the first group's settings ignored.

ie I've changed the test, the profile now looks like:

    [settings]
    arch=x86_64
    os=Linux
    build_type=Release
    compiler=clang
    compiler.libcxx=libstdc++11
    compiler.version=18
    compiler.cppstd=20

    [conf]
    tools.build:compiler_executables={'c': '/usr/bin/clang-18', 'cpp': '/usr/bin/clang++-18'}

    [settings]
    package/*:option=Whatever

Note that second [settings] group
The output is now:

E           conans.errors.ConanInvalidConfiguration: conanfile.py (hello/1.0): Invalid ID: Invalid: 'settings.os' value not defined

While I agree this is user-error, profile files can be long, and the cause is not obvious.
I think conan's handling of the profile files should change to either:
a) consider ALL items in all matching [group] sections
or, b) error if there is a repeated [group]

ie the user won't be aware that some of the profile settings will be completely ignored.

I've pushed an update to the test, not sure how to append to your PR, I'll mention it there

[memsharded]

Thanks for the feedback!

Now it makes sense what was the issue.

I agree that it would be better to have a clear error report of this situation. I think going with b) error if there is a repeated [group] would be the way to go, as having repeated sections sounds in general more like an unintended thing, and even if things are merged in sections the final behavior can be confusing to the user.

Lets try to add that error.

[paulharris]

Great, thanks!