Update documentation to clarify YAML loader design

Per feedback: YAML consolidation not required, fremor config format is sufficient.
- Update README to emphasize single-file YAML generated by fremor config
- Update cmor_yaml_consolidator docstrings to remove multi-file references
- Update YAML_SCHEMA_VALIDATION.md to clarify design intent
- Update code comments to reflect backward compatibility focus
- Remove references to multi-file consolidation as a feature

Agent-Logs-Url: https://github.com/ilaflott/fremorizer/sessions/70ef490b-d026-4828-ae72-ea094bf80d11

Co-authored-by: ilaflott <6273252+ilaflott@users.noreply.github.com>

Author: Claude

README.md

@@ -113,17 +113,13 @@ command now:
 
 - Uses native YAML loading and validation by default
 - Validates YAML configurations against a JSON schema
+- Works with the YAML format generated by `fremor config`
 - Automatically falls back to `fre-cli`'s `yamltools.consolidate_yamls` if available
-  (for advanced features like multi-file consolidation and platform/target overrides)
+  (for backward compatibility with legacy workflows)
 
-To use the advanced features provided by `fre-cli`, install it separately:
-```bash
-pip install fre-cli
-```
-
-**Note:** The native YAML loader provides a simplified version suitable for most
-use cases. For complex multi-file YAML consolidation with platform/target/experiment
-overrides, the `fre-cli` package is still recommended.
+The native YAML loader is designed to work with YAML files generated by
+`fremor config`, which produces self-contained configuration files that don't
+require multi-file consolidation or platform/target overrides.
 
 ## License
 

YAML_SCHEMA_VALIDATION.md

@@ -43,10 +43,10 @@ Provides native YAML loading and validation functionality.
 - Optional output file generation
 - Compatibility parameters for fre-cli signature
 
-**Limitations compared to fre-cli:**
-- Does not support multi-file YAML consolidation
-- Does not support platform/target/experiment-specific overrides
-- No variable substitution beyond environment variables
+**Design:**
+This module is designed to work with YAML files generated by `fremor config`,
+which produces self-contained configuration files. No multi-file consolidation
+or platform/target/experiment overrides are supported or needed.
 
 ### Modified Modules
 
@@ -56,7 +56,7 @@ Updated to use native YAML loading by default, with automatic fallback to fre-cl
 **Changes:**
 - Lines 27: Added import for `load_and_validate_yaml`
 - Lines 96-111: Replaced hard dependency with conditional logic:
-  - If `fre-cli` is available: Use `consolidate_yamls` (advanced features)
+  - If `fre-cli` is available: Use `consolidate_yamls` (for backward compatibility)
   - If `fre-cli` is NOT available: Use native `load_and_validate_yaml`
   - No more ImportError - both paths work
 
@@ -111,19 +111,24 @@ Comprehensive test suite covering:
 
 The implementation maintains 100% backward compatibility:
 
-1. **With fre-cli installed:** Uses `consolidate_yamls` (no change in behavior)
-2. **Without fre-cli installed:** Uses native loader (new functionality)
+1. **With fre-cli installed:** Uses `consolidate_yamls` (for backward compatibility)
+2. **Without fre-cli installed:** Uses native loader (standard approach)
 3. **Existing tests:** Continue to work with mocked `consolidate_yamls`
 4. **Configuration format:** Exactly the same structure expected
+5. **Generated YAML:** Works with output from `fremor config`
 
 ## Usage Examples
 
 ### Basic usage (native loader):
 ```bash
-fremor yaml model.yaml --exp test --platform ncrc4 --target prod
+# Generate a YAML configuration from a PP directory
+fremor config --pp-dir /path/to/pp --mip-era CMIP6 --output config.yaml ...
+
+# Use the generated YAML configuration
+fremor yaml config.yaml --exp test --platform ncrc4 --target prod
 ```
 
-### With advanced features (requires fre-cli):
+### With fre-cli (for backward compatibility):
 ```bash
 pip install fre-cli
 fremor yaml model.yaml --exp test --platform ncrc4 --target prod
@@ -140,25 +145,23 @@ config = load_and_validate_yaml('model.yaml')
 ## Migration Path
 
 For users currently using fre-cli:
-1. No changes required - fre-cli continues to work
-2. Optional: Remove fre-cli dependency for simpler workflows
-3. Optional: Update YAML files to be standalone (no multi-file inheritance)
+1. No changes required - fre-cli continues to work for backward compatibility
+2. Optional: Switch to `fremor config` to generate YAML files
+3. Optional: Remove fre-cli dependency if not needed for other workflows
 
 For new users:
 1. Install fremorizer: `pip install fremorizer`
-2. Create YAML config using `example_cmor_config.yaml` as template
-3. Run: `fremor yaml your_config.yaml`
+2. Generate YAML config: `fremor config --pp-dir /path/to/pp ...`
+3. Run CMORization: `fremor yaml config.yaml`
 4. No fre-cli installation needed
 
 ## Future Enhancements
 
 Potential improvements:
-1. Multi-file YAML consolidation in native loader
-2. Platform/target/experiment override support
-3. Enhanced variable substitution (beyond environment variables)
-4. YAML schema auto-generation from Python types
-5. Additional validation rules (path existence, value ranges)
-6. Schema versioning for CMIP6 vs CMIP7 differences
+1. Enhanced variable substitution (beyond environment variables)
+2. Additional validation rules (path existence, value ranges)
+3. Schema versioning for CMIP6 vs CMIP7 differences
+4. Support for YAML anchors/references in generated configs
 
 ## Testing Notes
 

fremorizer/cmor_yaml_consolidator.py

@@ -1,15 +1,16 @@
 """
-Native YAML Consolidation for CMOR Configuration
-================================================
+Native YAML Loader for CMOR Configuration
+==========================================
 
-This module provides native YAML consolidation functionality to replace
+This module provides native YAML loading functionality to replace
 the dependency on fre-cli's yamltools.consolidate_yamls function.
 
 Functions
 ---------
 - ``load_and_validate_yaml(...)`` - Load and validate a CMOR YAML configuration file
 
-.. note:: This module eliminates the dependency on fre-cli for YAML processing
+.. note:: This module eliminates the dependency on fre-cli for YAML processing.
+          It is designed to work with YAML files generated by fremor config.
 """
 
 import os
@@ -38,8 +39,11 @@ def load_and_validate_yaml(
     Load and validate a CMOR YAML configuration file.
 
     This function loads a YAML file, validates its structure against the CMOR schema,
-    and returns the configuration dictionary. This is a simplified replacement for
-    fre-cli's consolidate_yamls function that doesn't require external dependencies.
+    and returns the configuration dictionary. This is a replacement for fre-cli's
+    consolidate_yamls function that doesn't require external dependencies.
+
+    Designed to work with YAML files generated by 'fremor config', which produces
+    self-contained configuration files.
 
     :param yamlfile: Path to the YAML configuration file
     :type yamlfile: str
@@ -51,7 +55,7 @@ def load_and_validate_yaml(
     :type target: str, optional
     :param use: Configuration section to use (default: 'cmor')
     :type use: str
-    :param output: Optional output filename (for compatibility, currently unused)
+    :param output: Optional output filename to write validated config
     :type output: str, optional
     :raises FileNotFoundError: If the YAML file does not exist
     :raises ValueError: If the YAML structure is invalid
@@ -63,13 +67,6 @@ def load_and_validate_yaml(
         >>> config = load_and_validate_yaml('model.yaml')
         >>> print(config['cmor']['mip_era'])
         CMIP6
-
-    .. note:: This function provides a simplified version of YAML consolidation.
-              It does not support:
-              - Multi-file YAML consolidation
-              - Platform/target/experiment-specific overrides
-              - Variable substitution
-              For these features, install fre-cli and use its consolidate_yamls function.
     """
     # Expand environment variables in path
     yamlfile = os.path.expandvars(yamlfile)

fremorizer/cmor_yamler.py

@@ -91,8 +91,8 @@ def cmor_yaml_subtool( yamlfile: str = None,
     # ---------------------------------------------------
     # parsing the target model yaml ---------------------
     # ---------------------------------------------------
-    # Use native YAML consolidation by default, with optional fre-cli fallback
-    # for advanced features (multi-file consolidation, platform/target overrides)
+    # Use native YAML loader by default, with optional fre-cli fallback
+    # for backward compatibility with legacy workflows
     if consolidate_yamls is not None:
         fre_logger.info('using fre-cli consolidate_yamls for YAML processing')
         cmor_yaml_dict = consolidate_yamls(yamlfile=yamlfile,