Skip to content

Sets distro-specific OS name#24

Merged
bthomee merged 4 commits intomainfrom
bthomee/settings
Jul 30, 2025
Merged

Sets distro-specific OS name#24
bthomee merged 4 commits intomainfrom
bthomee/settings

Conversation

@bthomee
Copy link
Contributor

@bthomee bthomee commented Jul 30, 2025

Packages generally are configured to have the architecture, OS, and compiler+version determine the package ID. This change adds a new OS option per CI image to support the distro being used, e.g. "debian-bookworm".

Alternative options exist, e.g. https://docs.conan.io/2/reference/config_files/settings.html#customizing-settings. However, in practice this did not work after adding Linux-specific distros. Not only does it seem there's a current bug - the value "null" still resulted in errors in dependencies as they didn't have a distro defined, even though the use of "null" is precisely supposed to allow that - but the whole point is that dependencies need to be built for specific distros.

This change will ensure that each package in the dependency tree will see "os" set to the specific distro. You will now see things like:

docker run --rm -it -v ${CODEBASE}:/rippled ${CONTAINER_REGISTRY}/${CONTAINER_IMAGE}
[root@f51814e06059 ~]# conan profile show
Host profile:
[settings]
arch=armv8
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=13
os=rhel-9.6
[conf]
tools.build:compiler_executables={'c': '/usr/bin/gcc', 'cpp': '/usr/bin/g++'}

Build profile:
[settings]
arch=armv8
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=13
os=rhel-9.6
[conf]
tools.build:compiler_executables={'c': '/usr/bin/gcc', 'cpp': '/usr/bin/g++'}

(see the "os" fields)

Other changes:

  • During troubleshooting I found that a promising option had been added to Conan 2.19. While it didn't work, there are additional improvements, so I just left the update.
  • Some cleanup of comments.

@bthomee bthomee requested a review from Bronek July 30, 2025 13:09
@Bronek
Copy link
Collaborator

Bronek commented Jul 30, 2025

I guess this will require our CI pipelines to override the os from the default profile, e.g. with --settings os=debian-bookworm etc when we call conan install ?

@Bronek
Copy link
Collaborator

Bronek commented Jul 30, 2025

... it also means that anyone who wants to use our Conan artifactory will have to also set the os as appropriate, and hope that we have the matching artifacts. Which is not such a bad thing actually.

@bthomee
Copy link
Contributor Author

bthomee commented Jul 30, 2025

I guess this will require our CI pipelines to override the os from the default profile, e.g. with --settings os=debian-bookworm etc when we call conan install ?

Yes and no. The recipe hashes are not affected by the machine-specific settings, so anyone using the default os=Linux will still be able to find our (patched) recipes in our Artifactory. However, they'll not find any matching compiled packages in Artifactory, and would need to build all dependencies locally.

@Bronek
Copy link
Collaborator

Bronek commented Jul 30, 2025

I guess this will require our CI pipelines to override the os from the default profile, e.g. with --settings os=debian-bookworm etc when we call conan install ?

Yes and no. The recipe hashes are not affected by the machine-specific settings, so anyone using the default os=Linux will still be able to find our (patched) recipes in our Artifactory. However, they'll not find any matching compiled packages in Artifactory, and would need to build all dependencies locally.

That's good, on both accounts.

@Bronek
Copy link
Collaborator

Bronek commented Jul 30, 2025

I guess this will require our CI pipelines to override the os from the default profile, e.g. with --settings os=debian-bookworm etc when we call conan install ?

Actually I see that I was wrong, given that our CI pipelines would use the settings_user.yml which you added to the images.

os:
debian-${DEBIAN_VERSION}:
EOT
sed -i "s/os=Linux/os=debian-${DEBIAN_VERSION}/g" $(conan config home)/profiles/default
Copy link
Collaborator

@Bronek Bronek Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps we do not need these sed at all ? I assume settings_user.yml would take precedence ?

Copy link
Contributor Author

@bthomee bthomee Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The settings_user.yml tells Conan: "Please take the existing settings.yml and merge the settings in settings_user.yml into it" - see the link I had added in the PR description. It essentially allows you to expand or overwrite the available options.

Run for example cat $(conan config home)/settings.yml to see valid values for the profile.

Changing the available settings does not change the profile - so that's still set to os=Linux as a result of running the auto-detection, and it thus still needs to be overwritten.

@bthomee
Copy link
Contributor Author

bthomee commented Jul 30, 2025

Actually I see that I was wrong, given that our CI pipelines would use the settings_user.yml which you added to the images.

I believe you are actually correct - we can either update the value in the profile (e.g. using sed), or otherwise overwrite the value in the profile by passing the --setting os=[distro]-[version] flag. This still needs to be done in the CI pipelines for 'nix.

@Bronek
Copy link
Collaborator

Bronek commented Jul 30, 2025

Actually I see that I was wrong, given that our CI pipelines would use the settings_user.yml which you added to the images.

I believe you are actually correct - we can either update the value in the profile (e.g. using sed), or otherwise overwrite the value in the profile by passing the --setting os=[distro]-[version] flag. This still needs to be done in the CI pipelines for 'nix.

The profile that's created in the images created here is a throw-away anyway. The actual profile that we use when building rippled is in https://github.com/XRPLF/rippled/tree/develop/conan/profiles . I think clio team have something similar but generated by scripts, in https://github.com/XRPLF/clio/tree/develop/.github/scripts/conan

EDIT: In either case, --settings os .... will have to be used I think, to direct local Conan to look for the correct artifacts in the artifactory.

@bthomee bthomee merged commit 624dfa3 into main Jul 30, 2025
39 checks passed
@bthomee bthomee deleted the bthomee/settings branch July 30, 2025 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants