Optimize conda configuration: write .condarc directly instead of spawning subprocesses#453
Open
Optimize conda configuration: write .condarc directly instead of spawning subprocesses#453
Conversation
Replace ~26 sequential `conda config` subprocess calls with a single direct YAML file write using js-yaml. Each conda subprocess incurs 2-5s of Python/conda startup overhead, so this eliminates 60-130s of wall time for configuration alone. Key changes: - Replace applyCondaConfiguration with writeCondaConfig that builds the complete .condarc in memory and writes it once via fs.writeFileSync - Merge user condarc file (if provided) with action inputs in memory instead of copying then modifying via subprocess calls - Remove redundant second applyCondaConfiguration call after base tool installation (no longer needed since config is file-based) - Replace isDefaultEnvironment conda subprocess with local YAML parsing of user-level and prefix-level condarc files - Remove HTML index page download for Miniconda version validation; let the actual download fail with a clear HTTP error instead - Scope macOS chown -R to specific directories (bin, condabin, etc) instead of the entire conda tree, and run them in parallel - Parallelize Windows takeown calls with Promise.all - Remove get-hrefs and normalize-url dependencies (73 packages removed)
Address 8 stability concerns identified during review of the direct .condarc writing optimization: 1. Document channel priority order equivalence (comment) 2. Merge installer-written condarc as base instead of overwriting 3. Warn on unrecognized condarc keys (catches typos) 4. Strip 'defaults' from prefix-level condarc when removeDefaults set 5. Handle all YAML 1.1 boolean literals (on/off/y/n/1/0) 6. Bump minimum miniconda version to 25.5.0 (for auto_activate) 7. Revert macOS chown to full base path for safety 8. Improve error message when miniconda download fails Also switches condarc key from deprecated auto_activate_base to the canonical auto_activate (introduced in conda 25.5.0, old name removed in 26.3).
- Do not strip 'defaults' from channels when the user explicitly listed
it alongside 'nodefaults' (matches old behavior where conda config
--add would add defaults first, then the removeDefaults guard checked
`!channels.includes("defaults")` before removing).
- Bump Issue 261 test from miniconda py312_25.3.1-1 to py312_25.11.1-1
to satisfy the new >= 25.5.0 minimum version requirement.
Miniconda py312_25.11.1-1 doesn't publish macOS x86_64 builds. Remove the hardcoded architecture: x64 so the test uses the runner's native arch (arm64 on macOS, x64 on Linux/Windows).
jezdez
force-pushed
the
perf/optimize-conda-config
branch
from
d68e536 to
daa14fc
Compare
jezdez
force-pushed
the
perf/optimize-conda-config
branch
from
daa14fc to
a2fca81
Compare
3 tasks
Member
Author
|
@goanpeca Waiting to see if conda/conda#15741 is successful.. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
applyCondaConfiguration()(which spawned ~13 sequentialconda configsubprocesses per run, each paying 2-5s of Python startup) withwriteCondaConfig()that builds the complete.condarcin memory and writes it once viafs.writeFileSync.applyCondaConfigurationcall after base tool installation.conda config --show --jsonsubprocess inisDefaultEnvironmentwith direct YAML reads of condarc files.minicondaVersions()which downloaded and parsed the fullrepo.anaconda.comHTML index page for version validation. Now attempts the download directly with a clear error on failure.takeowncalls withPromise.all.Breaking changes
miniconda-versionbumped from 4.6.0 to 25.5.0 (required forauto_activateconfig key support).auto_activate(canonical name since conda 25.5.0) instead of the deprecatedauto_activate_base(removal scheduled for conda 26.3).Stability
~/.condarcwritten by the installer is now read as base config before applying action inputs, preserving installer-embedded settings (e.g., Miniforge'schannels: [conda-forge]).removeDefaultsnow also stripsdefaultsfrom prefix-level condarc files, matching the old behavior.on/off/y/n/1/0) in boolean config values.Tasks and Maintenance
get-hrefsandnormalize-urldependencies.src/typings.d.ts(only containedget-hrefstype declarations).