Merge pull request #32 from JGCRI/develop

Version 2.0.0-beta.6

Author: crvernon

.gitignore

@@ -9,6 +9,7 @@ notebooks/.ipynb_checkpoints/
 notebooks/beginner/.ipynb_checkpoints/
 notebooks/intermediate/.ipynb_checkpoints/
 notebooks/advanced/.ipynb_checkpoints/
+.scalable/
 
 # -----------------------------
 # Python bytecode / caches
@@ -110,4 +111,7 @@ communicator/src/communicator
 # -----------------------------
 # Existing repo-specific generated files
 # -----------------------------
-scalable/custom_types.py
+scalable/custom_types.py
+# Demeter tutorial artifacts (see docs/tutorials/demeter_setup.rst)
+/demeter_data/
+/outputs/

CHANGELOG.md

@@ -33,6 +33,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Documentation recommends beginner path**: `README.md`, `docs/getting_started.rst`,
   and `docs/tutorials/index.rst` now recommend starting with beginner tutorials for
   users unfamiliar with distributed computing concepts.
+- **New optional dependency extra**: added `[project.optional-dependencies] all`
+  to install test, dev, ai, cloud, kubernetes, and ml dependency groups in one
+  command (`pip install scalable[all]`).
+
+### Changed
+
+- Bumped package version to `2.0.0b2`.
+- Updated installation examples in `README.md` and `docs/getting_started.rst`
+  to use `pip install scalable[all]` for all optional dependencies.
 
 ---
 

CITATION.cff

@@ -2,7 +2,7 @@ cff-version: 1.2.0
 message: "If you use this software, please cite it as below."
 type: software
 title: "Scalable"
-version: "2.0.0-beta.1"
+version: "2.0.0-beta.6"
 doi: "10.5281/zenodo.20295640"
 url: "https://github.com/JGCRI/scalable"
 repository-code: "https://github.com/JGCRI/scalable"

README.md

@@ -24,7 +24,6 @@ Scalable is a Python framework for orchestrating containerized, distributed work
   - [Telemetry and Reports](#telemetry-and-reports)
   - [Resource Advising](#resource-advising)
   - [ML Optimization](#ml-optimization)
-  - [Model Emulation](#model-emulation)
   - [AI Assistants](#ai-assistants)
   - [Cloud and Kubernetes](#cloud-and-kubernetes)
   - [Artifact Storage](#artifact-storage)
@@ -120,11 +119,11 @@ pip install scalable[cloud]
 # Kubernetes provider (Dask Kubernetes Operator)
 pip install scalable[kubernetes]
 
-# ML optimization and emulation (LearnedAdvisor, AdaptiveScaler, emulators)
+# ML optimization (LearnedAdvisor, AdaptiveScaler)
 pip install scalable[ml]
 
 # All optional dependencies
-pip install scalable[ai,cloud,kubernetes,ml]
+pip install scalable[all]
 ```
 
 If your shell cannot find installed scripts (for example, `scalable_bootstrap`), add the relevant scripts directory to `PATH`.
@@ -372,34 +371,6 @@ CLI access:
 scalable advise --task run_gcam --model-type gradient_boosting --format json
 ```
 
-### Model Emulation
-
-The emulation subsystem (`scalable[ml]`) provides uncertainty-aware surrogate
-model dispatch for expensive scientific functions:
-
-```python
-from scalable import emulatable, EmulatorRegistry, EmulatorDispatch
-
-@emulatable(
-    inputs=["temperature", "precipitation"],
-    outputs=["yield"],
-    domain_bounds={"temperature": (250, 350), "precipitation": (0, 5000)},
-    confidence_threshold=0.9,
-)
-def run_crop_model(temperature, precipitation):
-    # Expensive model execution
-    ...
-
-# Register and manage trained emulators
-registry = EmulatorRegistry(".scalable/emulators")
-dispatch = EmulatorDispatch(registry, confidence_threshold=0.9)
-
-# Confidence-gated routing: uses emulator when confident, falls back to full model
-result = dispatch.predict("run_crop_model", inputs={"temperature": 300, "precipitation": 1200})
-print(result.source)       # "emulator" or "full_model"
-print(result.confidence)
-```
-
 ### AI Assistants
 
 AI assistants help with onboarding, diagnostics, workflow generation, and
@@ -630,9 +601,6 @@ Override `AI_API_KEY` for individual providers when using multiple services:
 | `SCALABLE_DEFAULT_STORAGE` | *(unset)* | Default artifact storage URI |
 | `SCALABLE_ML` | `1` | Enable ML features |
 | `SCALABLE_ML_CACHE_DIR` | `.scalable/models` | ML model cache directory |
-| `SCALABLE_EMULATION` | `0` | Enable model emulation |
-| `SCALABLE_EMULATOR_DIR` | `.scalable/emulators` | Emulator registry directory |
-| `SCALABLE_EMULATION_CONFIDENCE` | `0.9` | Emulation confidence threshold |
 
 ### Advanced AI Overrides
 

docs/advising.rst

@@ -14,7 +14,7 @@ Quick start
 
     advisor = ResourceAdvisor.from_history("./.scalable/runs")
     recommendation = advisor.recommend(
-        task="run_gcam",
+        task="run_demeter_scenario",
         target="local",
         confidence=0.95,
     )
@@ -42,7 +42,7 @@ CLI access
 
 .. code-block:: bash
 
-    scalable advise --task run_gcam --target local --confidence 0.95
+    scalable advise --task run_demeter_scenario --target local --confidence 0.95
 
 The CLI ``advise`` command first attempts ML-backed recommendations (if
 ``scalable[ml]`` is installed) and falls back to the heuristic advisor when

docs/ai_assistants.rst

@@ -1,9 +1,9 @@
 AI Assistants
 =============
 
-Scalable v2.0.0 includes AI-assisted features (Phase 4) that help users
-onboard models, compose workflows, diagnose failures, explain plans, and
-migrate manifests.
+Scalable v2.0.0 includes AI-assisted features that help users onboard
+models, compose workflows, diagnose failures, explain plans, and migrate
+manifests.
 
 All features work **without** an LLM backend via deterministic heuristic
 fallbacks. LLM enhancement is opt-in via the ``SCALABLE_AI_BACKEND``
@@ -162,17 +162,15 @@ scalable compose
 
 Generate a workflow from a natural-language description::
 
-    scalable compose "Run GCAM reference scenario then Stitches for daily weather"
-    scalable compose "Run Hector model" --output-dir ./generated
+    scalable compose "Run the GCAM reference scenario and downscale land allocation using Demeter" --output-dir ./generated
+    scalable compose "Run the GCAM reference scenario" --output-dir ./generated
 
 Options:
 
 * ``--output-dir`` — Directory for generated files
 * ``--format`` — Output format (``text`` or ``json``)
 * ``--no-ai`` — Use heuristics only
 
-Known model patterns: GCAM, Stitches, Demeter, Tethys, Xanthos, Hector.
-
 scalable migrate
 ~~~~~~~~~~~~~~~~
 
@@ -264,5 +262,4 @@ See Also
 
 - :doc:`manifest` — Manifest schema and session API
 - :doc:`ml` — ML-backed resource optimization
-- :doc:`emulation` — Model emulation with surrogate dispatch
 - :doc:`telemetry` — Run telemetry that powers diagnosis

docs/artifacts.rst

@@ -1,9 +1,9 @@
 Artifact Store
 ==============
 
-The :mod:`scalable.artifacts` module (Phase 3) provides a protocol-based
-abstraction for storing and retrieving workflow artifacts across local and
-remote backends.
+The :mod:`scalable.artifacts` module provides a protocol-based abstraction
+for storing and retrieving workflow artifacts across local and remote
+backends.
 
 Overview
 --------

docs/cloud.rst

@@ -45,8 +45,8 @@ GCP Provider (Scaffold)
 -----------------------
 
 The :class:`~scalable.providers.cloud.gcp.GCPProvider` is a validation-only
-scaffold in Phase 3. It validates manifest options but raises
-``NotImplementedError`` on ``build_cluster()``.
+scaffold. It validates manifest options but raises ``NotImplementedError``
+on ``build_cluster()``.
 
 Target options: