feat: Add Support for Rattler-Build Build Backend - #1127
Open
seneschall wants to merge 26 commits into
Open
Conversation
…d_recipes in tests
…ls into rattler_support2
…raph.build as methods of MetaOrRattler
utils.parallel_iter was called on a function which forced it to pickle rattler_build.VariantConfig, which resulted in failure to build
…ler_args; minor fixes
Accidentally added rattler-build args instead of conda-build args prior to this.
In single_build, the config must be loaded, otherwise utils.RepoData can't be instantiated. Also the recipe path needed to be converted to a utils.RecipePath.
Author
|
I have done my best to merge with the master branch but as the two were quite divergent it is possible I missed something and caused breaking changes. I will not have time to work on this PR until next week and I didn't get the chance to test the merged code locally yet so please be patient. But if you want you can start reviewing this PR regardless. I will answer you as soon as possible. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for the rattler-build backend which uses the v1 recipe syntax. In order to enable this, I had to make changes in a lot of places in the code base since rattler-build handles a lot of things differently from conda-build. One important change I made e.g. was to introduce a
RecipePathclass which stores the build-backend for every recipe alongside the path that points to it. This way the rest of the code can automatically determine which build-backend to use for every given recipe based on whether the recipe folder contains ameta.yamlor arecipe.yaml.During these changes I also added type hints wherever possible in order to better understand the existing codebase and in order to catch type errors via the linter instead of at runtime. Wherever possible, I also changed string based paths to pathlib Path objects to both improve code readability by clearly marking strings that are supposed to be paths and to make the manipulation of paths safer and more idiomatic.
Also, I deprecated the
testonlyflag because from my understanding of the existing code, bioconda-utils expects to be able to callconda-build --teston the recipe directory. This definitely doesn't work forrattler-build test, as rattler-build expects to be passed a directory of already built packages, but from what I understandconda-builddoes the same, as per this excerpt from their help: "RECIPE_PATH argument must be a path to built package file". Also, there don't seem to be any existing test cases wheretestonlyis set toTrue. If I am mistaken, please feel free to revert this change.I ran all tests from
test/test_utils.pyandtest/test_recipe.pylocally, but I only tested for native builds and not for docker based builds. The parts of the code that handle the docker build process should thus be examined more closely. The tests all passed except fortest_multi_build. This one also fails on my machine for the version from themasterbranch, however. I also added a single test case,test/test_utils.py::test_rattler_recipe.Parts of the code where I was unsure about something or which should be improved, I have marked with
TODO (rb). If you have any questions about my code, don't hesitate to ask.To summarise:
testonlyflag, as it seems to be broken as far as I understand and it also isn't covered by any of the existing test cases