Agent/scientific workflow contracts - #5
Open
allaffa wants to merge 43 commits into
Open
Conversation
- backends/dft/__init__.py: re-export DFTBackend, DFTJobSpec, DFTResult (Protocol already existed in active_learning/dft_backend.py) - backends/mlip/__init__.py: new @runtime_checkable MLIPBackend Protocol with name, as_calculator() -> Calculator, relax() -> RelaxationResult - backends/llm/__init__.py: LLMBackend = BaseChatModel (TYPE_CHECKING alias) - execution/__init__.py: new @runtime_checkable ExecutionPlatform Protocol with name, submit() -> str, available_resources() -> ResourceRequest - execution/provenance.py: new @runtime_checkable RunStore Protocol + concrete JsonlRunStore implementation - tests/test_package_boundaries.py: 11 tests covering all five interfaces, Protocol compliance stubs, and JsonlRunStore round-trip
- README.md project layout: replace flat src/ tree with new
orchestration/, backends/{llm,mlip,dft}/, and execution/ packages;
mark state.py / graph.py / llm.py / tools/relaxation.py as compat
aliases; add vasp_runner.py / orchestration/policies/ / execution/
launchers & provenance
- README.md programmatic API: update import paths to canonical modules
tools.relaxation → backends.mlip.relaxation
matsim_agents.graph → orchestration.objective_graph
matsim_agents.state → orchestration.state
Update standalone DFT relaxer table to link backends/dft/ modules
- README.md configuration reference: link to backends/mlip/relaxation.py
- docs/architecture.md: add 'Five stable interfaces' table documenting
DFTBackend, MLIPBackend, LLMBackend, ExecutionPlatform, RunStore
matsim_agents.llm → backends.llm matsim_agents.tools.vasp_relax → backends.dft.vasp_relax (×2)
tools.vasp_relax → backends.dft.vasp_relax tools.qe_relax → backends.dft.qe_relax
The competition bundle is a reusable benchmark suite with documented inputs (159 structures, 5 tasks) and outputs — a better fit for benchmarks/ than research/. Update all README.md references.
cfg.hydragnn.logdir → cfg.mlip.hydragnn.logdir data['mlp'] → data['mlip'] (×4 tests) regex 'mlp.uma' → 'mlip.uma', 'mlp.hydragnn' → 'mlip.hydragnn'
allaffa
added
documentation
There was a problem hiding this comment.
🟡 Changes recommended
There are correctness/documentation issues in the new deployment guidance and validation logic (including a Slurm-spool PROJECT_ROOT resolution bug) that should be fixed before merging.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR introduces “scientific workflow contracts” and reorganizes the codebase around clearer, stable boundaries (orchestration/, execution/, backends/) while migrating machine-specific assets into deployments/ and publication artifacts into research/. It also adds tests that assert key safety and governance invariants (approvals, provenance, dataset reference consistency, and deployment portability).
Changes:
- Add execution/run contracts (provenance records, run directory layout, resource/launcher protocols) and new CLI workflow entrypoints.
- Refactor imports and package layout to explicit
orchestration/*state/graphs andbackends/*implementations, keeping compatibility aliases. - Migrate HPC scripts and documentation to
deployments/and paper artifacts toresearch/, with new validation tests and portability checks.
File summaries
| File | Description |
|---|---|
| tests/test_workflow_policies.py | Adds tests for AL/workflow policy invariants. |
| tests/test_scientific_run_contracts.py | Tests run directory + workflow result contract validation. |
| tests/test_open_model_catalog.py | Validates open-model catalog consistency across deployment assets. |
| tests/test_deployments.py | Runs static deployment portability validation in CI. |
| tests/test_dataset_energy_references.py | Tests dataset append rejects mixed DFT energy references. |
| tests/test_al_mlip_config.py | Updates config tests for mlip renaming and validation strings. |
| tests/test_al_config.py | Adjusts AL config test to new cfg.mlip.* access path. |
| tests/integration/test_uma_warmstart.py | Updates docstring paths to migrated deployment scripts. |
| tests/integration/test_uma_vasp_warmstart.py | Updates docstring paths to migrated deployment scripts. |
| tests/integration/test_qe_warmstart.py | Updates docstring paths to migrated deployment scripts. |
| tests/integration/test_mace_warmstart.py | Updates docstring paths to migrated deployment scripts. |
| tests/integration/test_mace_vasp_warmstart.py | Updates docstring paths to migrated deployment scripts. |
| src/matsim_agents/workflows/init.py | Exposes workflow contract API surface via __all__. |
| src/matsim_agents/tools/warmstart_benchmark_vasp.py | Redirects imports to new backend module locations. |
| src/matsim_agents/tools/warmstart_benchmark_qe.py | Redirects imports to new backend module locations. |
| src/matsim_agents/tools/init.py | Re-exports relaxation tool via new backend path. |
| src/matsim_agents/state.py | Converts legacy state module to compatibility alias. |
| src/matsim_agents/orchestration/state.py | New canonical typed orchestration state module. |
| src/matsim_agents/orchestration/policies/init.py | Adds orchestration policy package marker/docs. |
| src/matsim_agents/orchestration/init.py | Exposes orchestration state types via __all__. |
| src/matsim_agents/execution/resources.py | Adds scheduler-neutral ResourceRequest. |
| src/matsim_agents/execution/provenance.py | Adds JSONL provenance helpers + RunStore protocol. |
| src/matsim_agents/execution/launchers.py | Adds launcher protocol interface. |
| src/matsim_agents/execution/init.py | Public execution API re-exports + ExecutionPlatform protocol. |
| src/matsim_agents/discovery/wrapper.py | Updates imports to orchestration state + backend relaxation. |
| src/matsim_agents/cli.py | Adds relax command + reroutes orchestration imports. |
| src/matsim_agents/chat.py | Moves LLM + relaxation imports to backends/*. |
| src/matsim_agents/backends/mlip/init.py | Defines MLIP backend protocol and exports relaxation entrypoints. |
| src/matsim_agents/backends/llm/init.py | Adds LLM backend accessor/export surface. |
| src/matsim_agents/backends/dft/init.py | Re-exports DFT backend protocol/types from AL layer. |
| src/matsim_agents/backends/init.py | Adds backends package marker. |
| src/matsim_agents/agents/planner.py | Updates state + LLM provider import paths. |
| src/matsim_agents/agents/executor.py | Updates state + relaxation import paths. |
| src/matsim_agents/agents/analyst.py | Updates state + LLM provider import paths. |
| src/matsim_agents/active_learning/vasp_runner.py | Moves VASP backend import to backends.dft. |
| src/matsim_agents/active_learning/trainer.py | Enforces single DFT backend reference per extxyz dataset append. |
| src/matsim_agents/active_learning/seeds.py | Updates LLM provider import path. |
| src/matsim_agents/active_learning/dft_backend.py | Updates DFT backend factory imports to backends.dft.*. |
| src/matsim_agents/active_learning/calculator.py | Updates HydraGNN calculator helpers import path. |
| src/matsim_agents/init.py | Updates public MatSimState export path. |
| scripts/diagnostics/validate_deployments.py | Adds static checks for deployment portability/safety. |
| scripts/diagnostics/render_paper_tables.py | Updates paper artifact paths from docs/ to research/. |
| scripts/diagnostics/collect_paper_results.py | Updates paper artifact output paths to research/. |
| scripts/advanced/common/runtime-env.sh | Removes legacy runtime env helper (migrated to deployments). |
| research/README.md | Adds research-only tree README. |
| research/paper/manuscript/tables/warmstart_matrix.tex | Adds paper table artifact. |
| research/paper/manuscript/figures/plot_dft_scaling.py | Updates input/output paths to research/. |
| research/paper/manuscript/figures/plot_al_timings.py | Updates run-from path to research/. |
| research/paper/manuscript/figures/plot_al_finetune.py | Updates run-from path to research/. |
| pyproject.toml | Adds ruff per-file ignores for new modules. |
| examples/relaxation/scientific_relaxation.example.yaml | Adds example config for scientific relaxation workflow. |
| examples/paper_cases/singlepass.py | Updates Perlmutter setup path to deployments/. |
| examples/paper_cases/README.md | Updates deployment paths + adds explicit promotion approvals in examples. |
| examples/paper_cases/al_zn_formate.yaml | Migrates launchers paths; adds model promotion approvals. |
| examples/paper_cases/al_zn_formate_uma.yaml | Migrates launcher paths to deployments/. |
| examples/paper_cases/al_phosphorene.yaml | Migrates launcher paths; adds model promotion approvals. |
| examples/paper_cases/al_phosphorene_qe.yaml | Migrates QE launcher path to deployments/. |
| examples/paper_cases/al_phosphorene_qe_frontier.yaml | Migrates Frontier launcher paths to deployments/. |
| examples/paper_cases/al_phosphorene_frontier.yaml | Migrates Frontier job/launcher paths; adds promotion approvals. |
| examples/paper_cases/al_lifepo4.yaml | Migrates launcher paths; adds model promotion approvals. |
| examples/paper_cases/al_lifepo4_frontier.yaml | Migrates Frontier job/launcher paths; adds promotion approvals. |
| examples/paper_cases/al_hea_fcc_scaling.yaml | Migrates job/launcher paths to deployments/. |
| examples/paper_cases/al_hea_fcc_scaling_frontier.yaml | Migrates Frontier job/launcher paths to deployments/. |
| examples/paper_cases/al_hea_fcc_cantor.yaml | Migrates launcher paths; adds promotion approvals. |
| examples/paper_cases/al_hea_bcc.yaml | Migrates launcher paths; adds promotion approvals. |
| examples/paper_cases/al_hea_bcc_frontier.yaml | Migrates Frontier job/launcher paths; adds promotion approvals. |
| examples/paper_cases/al_cu_bht_2d.yaml | Migrates launcher paths; adds promotion approvals. |
| examples/active_learning/README.md | Migrates setup/launcher paths to deployments/. |
| examples/active_learning/al_config.prompt.example.yaml | Migrates launcher paths; adds promotion approvals. |
| examples/active_learning/al_config.example.yaml | Migrates launcher paths; adds promotion approvals. |
| examples/active_learning_uq.py | Migrates Aurora QE launcher path to deployments/. |
| docs/workflows/perlmutter-smoke-summary-2026-06-12.txt | Updates smoke script path to deployments/. |
| docs/vasp-aurora.md | Migrates Aurora build/setup references to deployments/. |
| docs/quantum-espresso-frontier.md | Migrates Frontier setup/launcher references to deployments/. |
| docs/model-download.md | Migrates Perlmutter download/job references to deployments/. |
| docs/model-download-safety.md | Migrates all platform download/setup references to deployments/. |
| docs/hpc-validation.md | Adds cross-platform validation policy/matrix doc. |
| docs/architecture.md | Adds package architecture + compatibility policy doc. |
| deployments/README.md | Adds deployments policy + submission contract. |
| deployments/perlmutter/smoke-tests/smoke-transformers-perlmutter.sh | Removes embedded account/paths; uses PROJECT_ROOT; updates module stack path. |
| deployments/perlmutter/smoke-tests/smoke-transformers-multinode-perlmutter.sh | Removes embedded account/paths; uses PROJECT_ROOT; updates module stack path. |
| deployments/perlmutter/setup/smoke_qe_vasp_perlmutter.sh | Updates usage docs to deployments/ paths. |
| deployments/perlmutter/setup/job_perlmutter.sh | Removes embedded account directive. |
| deployments/perlmutter/setup/build-scalapack-perlmutter.sh | Updates stack path to deployments/. |
| deployments/perlmutter/setup/build-qe-gpu-perlmutter.sh | Removes embedded account/paths; uses PROJECT_ROOT; updates docs. |
| deployments/perlmutter/setup/build-qe-cpu-perlmutter.sh | Removes embedded account/paths; uses PROJECT_ROOT; updates docs. |
| deployments/perlmutter/setup/build-mace-venv-perlmutter.sh | Updates docs path to deployments/. |
| deployments/perlmutter/launchers/submit-finetune-eval-pack4-hydragnn-perlmutter.sh | Removes embedded fallback path; requires PROJECT_ROOT. |
| deployments/perlmutter/launchers/run-vasp-gpu-perlmutter.sh | Removes embedded fallback path; updates stack path to deployments/. |
| deployments/perlmutter/launchers/run-pw-gpu-perlmutter.sh | Removes embedded fallback path; updates stack path and srun flags. |
| deployments/perlmutter/launchers/launch-test-singlenode-resume-perlmutter.sh | Updates smoke script path + PROJECT_ROOT handling. |
| deployments/perlmutter/launchers/launch-test-multinode-perlmutter.sh | Updates smoke script path + PROJECT_ROOT handling. |
| deployments/perlmutter/launchers/launch-test-all-models-perlmutter.sh | Updates smoke script path + PROJECT_ROOT handling. |
| deployments/perlmutter/launchers/job-finetune-eval-perlmutter.sh | Removes embedded account; updates paths to deployments/. |
| deployments/perlmutter/launchers/job-finetune-eval-pack4-perlmutter.sh | Removes embedded account; updates paths to deployments/. |
| deployments/perlmutter/launchers/_vasp-step-perlmutter.sh | Requires PROJECT_ROOT; updates stack path to deployments/. |
| deployments/perlmutter/launchers/_qe-step-perlmutter.sh | Requires PROJECT_ROOT; updates stack path to deployments/. |
| deployments/perlmutter/launchers/_mace-train-step-perlmutter.sh | Requires PROJECT_ROOT; updates stack path to deployments/. |
| deployments/perlmutter/jobs/submit-vasp-warmstart-check-matrix.sh | Updates job/launcher paths to deployments/. |
| deployments/perlmutter/jobs/submit-uma-warmstart-check-matrix.sh | Updates job/launcher paths to deployments/. |
| deployments/perlmutter/jobs/submit-uma-vasp-warmstart-check-matrix.sh | Updates job/launcher paths to deployments/. |
| deployments/perlmutter/jobs/submit-qe-warmstart-check-matrix.sh | Updates job/launcher paths to deployments/. |
| deployments/perlmutter/jobs/submit-al-dft-scaling-sweep.sh | Updates references to research/ figure path. |
| deployments/perlmutter/jobs/submit-active-learning-paper-cases.sh | Updates references to deployments/ wrapper path. |
| deployments/perlmutter/jobs/job-vllm-smoke-perlmutter.sh | Removes embedded account; requires PROJECT_ROOT. |
| deployments/perlmutter/jobs/job-vasp-warmstart-perlmutter.sh | Removes embedded account; requires PROJECT_ROOT; updates launcher/stack paths. |
| deployments/perlmutter/jobs/job-singlepass-paper-cases-perlmutter.sh | Removes embedded account/paths; requires PROJECT_ROOT; updates setup path. |
| deployments/perlmutter/jobs/job-single-relaxation-perlmutter.sh | Removes embedded account; adjusts GPU request; updates CLI option name. |
| deployments/perlmutter/jobs/job-qe-warmstart-perlmutter.sh | Removes embedded account; requires PROJECT_ROOT; updates launcher/stack paths. |
| deployments/perlmutter/jobs/job-hypothesis-debate-perlmutter.sh | Switches to shared runtime-env helper; updates script paths. |
| deployments/perlmutter/jobs/job-discovery-vllm-perlmutter.sh | Switches to shared runtime-env helper; updates stack path. |
| deployments/perlmutter/jobs/job-discovery-chat-perlmutter.sh | Switches to shared runtime-env helper; updates stack path. |
| deployments/perlmutter/jobs/job-al-qe-portability-perlmutter.sh | Removes embedded account; requires PROJECT_ROOT; updates stack path. |
| deployments/perlmutter/jobs/job-al-dft-scaling-perlmutter.sh | Removes embedded account; requires PROJECT_ROOT; updates stack path. |
| deployments/perlmutter/jobs/job-active-learning-uq-perlmutter.sh | Removes embedded account/paths; requires PROJECT_ROOT; updates launcher path. |
| deployments/perlmutter/jobs/hypothesis_debate_beforeafter.py | Removes embedded site-specific default runs path. |
| deployments/perlmutter/download/download-uma-perlmutter.sh | Removes embedded account/paths; requires PROJECT_ROOT; updates stack path. |
| deployments/frontier/utils/_rocr_to_hip.sh | Adds helper for Ray/vLLM HIP device env compatibility. |
| deployments/frontier/smoke-tests/smoke-transformers-frontier.sh | Removes embedded account/paths; requires PROJECT_ROOT; updates stack path. |
| deployments/frontier/setup/README.md | Updates paths to deployments/ download/setup scripts. |
| deployments/frontier/setup/prebuild-tvm-ffi-frontier.sh | Removes embedded account/paths; requires PROJECT_ROOT; updates stack path. |
| deployments/frontier/setup/makefile.include.frontier-gpu | Updates comment paths to deployments/. |
| deployments/frontier/setup/install-rocm72.sh | Removes embedded absolute invocation; updates protected-reqs path. |
| deployments/frontier/setup/install_matsim_frontier.sh | Updates comments to deployments/ build paths. |
| deployments/frontier/setup/frontier-module-stack.sh | Adds shared Frontier module-stack helper. |
| deployments/frontier/setup/build-vllm-rocm72.sh | Removes embedded account/paths; requires PROJECT_ROOT; uses module-stack helper. |
| deployments/frontier/setup/build-vasp-gpu-frontier.sh | Removes embedded account/paths; updates docs to deployments/. |
| deployments/frontier/setup/build-qe-gpu-frontier.sh | Removes embedded account/paths; updates docs to deployments/. |
| deployments/frontier/setup/build-qe-cpu-frontier.sh | Removes embedded account/paths; removes log-dir mkdir hack. |
| deployments/frontier/setup/build-mace-venv-frontier.sh | Updates docs path to deployments/. |
| deployments/frontier/launchers/run-active-learning-frontier.sh | New/updated AL launcher with deployment-root resolution logic. |
| deployments/frontier/launchers/launch-test-singlenode-resume-frontier.sh | Requires PROJECT_ROOT; updates repo resolution. |
| deployments/frontier/launchers/launch-test-multinode-frontier.sh | Requires PROJECT_ROOT; updates repo resolution. |
| deployments/frontier/launchers/launch-test-all-models-frontier.sh | Requires PROJECT_ROOT; updates repo resolution. |
| deployments/frontier/launchers/_qe-step-frontier.sh | Updates comment path to deployments/. |
| deployments/frontier/launchers/_hydragnn-train-step-frontier.sh | Adds inner-step HydraGNN training launcher for AL. |
| deployments/frontier/jobs/submit-al-dft-scaling-sweep.sh | Updates references to research/ figure path. |
| deployments/frontier/jobs/job-six-model-benchmark-frontier.sh | Removes embedded account/paths; uses module-stack helper. |
| deployments/frontier/jobs/job-singlepass-paper-cases-frontier.sh | Removes embedded account/paths; requires PROJECT_ROOT; updates setup path. |
| deployments/frontier/jobs/job-single-relaxation-frontier.sh | Removes embedded account/paths; updates CLI option name. |
| deployments/frontier/jobs/job-serve-multinode-frontier.sh | Updates usage examples and setup references to deployments/. |
| deployments/frontier/jobs/job-qe-warmstart-frontier.sh | Removes embedded account/paths; updates launcher/stack paths. |
| deployments/frontier/jobs/job-discovery-chat-vllm-frontier.sh | Updates CLI option name; updates stack helper path. |
| deployments/frontier/jobs/job-al-dft-scaling-frontier.sh | Removes embedded account/paths; updates setup path. |
| deployments/frontier/jobs/job-active-learning-uq-frontier.sh | Removes embedded account/paths; updates launcher/stack paths. |
| deployments/frontier/jobs/job-active-learning-paper-cases-frontier.sh | Removes embedded account/paths; updates setup path. |
| deployments/frontier/docs/README-six-model-benchmark.md | Adds Frontier benchmark guide (currently has stale paths). |
| deployments/common/runtime-env.sh | Adds shared repo/run-dir resolution helper for spooled schedulers. |
| deployments/aurora/setup/setup_matsim_aurora.sh | Adds Aurora environment setup helper script. |
| deployments/aurora/setup/install-vllm-xpu-aurora.sh | Requires PROJECT_ROOT; updates next-step instructions to deployments/. |
| deployments/aurora/setup/hf_download.py | Makes Aurora HF downloader path-neutral and configurable. |
| deployments/aurora/setup/build-vasp-gpu-aurora.sh | Updates docs path to deployments/. |
| deployments/aurora/setup/build-qe-gpu-aurora.sh | Updates docs path to deployments/. |
| deployments/aurora/setup/build-qe-cpu-aurora.sh | Updates docs path to deployments/. |
| deployments/aurora/launchers/run-pw-gpu-aurora.sh | Switches to mpiexec-only flow; updates GPU bind default. |
| deployments/aurora/jobs/job-step2-perturbation-aurora.sh | Removes embedded account/paths; requires PROJECT_ROOT. |
| deployments/aurora/jobs/job-single-relaxation-aurora.sh | Removes embedded account/paths; updates CLI option name. |
| deployments/aurora/jobs/job-serve-multinode-aurora.sh | Removes embedded account/paths; updates references to deployments/. |
| deployments/aurora/jobs/job-qe-warmstart-aurora.sh | Removes embedded account/paths; updates launcher path. |
| deployments/aurora/jobs/job-discovery-chat-rhea-aurora.sh | Removes embedded account/paths; updates entrypoint path + CLI option name. |
| deployments/aurora/jobs/job-discovery-chat-perovskites-aurora.sh | Removes embedded account/paths; updates entrypoint path + CLI option name. |
| deployments/aurora/jobs/job-discovery-chat-aurora.sh | Switches to shared runtime-env helper; updates CLI option name. |
| deployments/aurora/jobs/job-active-learning-uq-aurora.sh | Removes embedded account/paths; updates launcher path. |
| benchmarks/README.md | Adds benchmarks tree README. |
| benchmarks/codabench/starting_kit/example_submission/task5.csv | Adds example submission artifact. |
| benchmarks/codabench/starting_kit/example_submission/task2.zip | Adds example submission archive artifact. |
| benchmarks/codabench/starting_kit/example_submission/task1.csv | Adds example submission artifact. |
| benchmarks/codabench/scoring_program/metadata | Adds Codabench scoring manifest. |
| benchmarks/codabench/requirements.txt | Adds Codabench runtime requirements. |
| benchmarks/codabench/reference_data/public_ids.txt | Adds public split IDs list. |
| benchmarks/codabench/pages/terms.md | Adds competition terms page. |
| benchmarks/codabench/pages/overview.md | Adds competition overview page. |
| benchmarks/codabench/pages/evaluation.md | Adds submission + evaluation spec page. |
| benchmarks/codabench/pages/data.md | Adds data description page. |
| benchmarks/codabench/fix_h5py_system_conflict_aurora.sh | Adds Aurora h5py conflict remediation helper. |
| benchmarks/codabench/baselines/mace_mp0/model.py | Adds MACE baseline implementing competition interface. |
| benchmarks/codabench/baselines/hydragnn/model.py | Adds HydraGNN baseline implementing competition interface. |
Review details
Suppressed comments (1)
deployments/frontier/launchers/run-active-learning-frontier.sh:36
PROJECT_ROOTdefaults to a path derived fromBASH_SOURCE[0], which points at Slurm's spool copy undersbatch. In that common case../../..is not the repo root, and the current check does not fail becausePROJECT_ROOTis non-empty, so the script will run with an invalid repo path and later fail in confusing ways. Prefer resolving viadeployments/common/runtime-env.sh(which already handlesSLURM_SUBMIT_DIR/PROJECT_ROOT) and then validate thatpyproject.tomlexists.
- Files reviewed: 222/273 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+50
to
+55
| if path.suffix == ".sh": | ||
| result = subprocess.run( | ||
| ["bash", "-n", str(path)], capture_output=True, text=True, check=False | ||
| ) | ||
| if result.returncode: | ||
| errors.append(f"{rel}: bash -n failed: {result.stderr.strip()}") |
…flow-contracts branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.