Develop Comprehensive Curryer Config Strategy [DRAFT ARCHITECTURE VISION]

[mmaclay]

Explore a Comprehensive Curryer Configuration Class Ecosystem (Draft issue, we should discuss as a team)

This is a draft/vision for where Curryer's configuration architecture could go. The goal is to spark discussion and guide collaboration on a modern, extensible config system (likely using Pydantic), to address pain points of environment variable sprawl and repeated parameters, and to future-proof for mission- and platform-specific needs.

The correction refactor #122 & #131 is the first concrete implementation of this vision.

---

Draft Idea: Configuration Architecture for Curryer

Pydantic

• Validation at runtime with helpful messages
• Type safety and IDE support
• Hierarchical/nested configs (composition/inheritance)
• Easy override for different missions/platforms
• Environment variable support
• Self-documenting

Example: Proposed Class Hierarchy (as an example)

from pydantic import BaseModel, Field
from pathlib import Path
from typing import Optional, List, Dict, Literal

class PlatformConfig(BaseModel):
    """Platform-specific settings (AWS, Local, HPC, etc.)"""
    data_dir: Path = Field(..., description="Root data directory")
    temp_dir: Path = Field(..., description="Directory for temporary files")
    storage_backend: Literal['local', 'aws', 'hpc'] = Field('local', description="Storage backend type")
    parallel_workers: int = Field(4, description="Number of parallel workers")

class PathConfig(BaseModel):
    """Path resolution and directory configuration"""
    kernel_dir: Path = Field(..., description="Kernel files directory")
    data_dir: Path = Field(..., description="Data directory")
    output_dir: Path = Field(..., description="Output directory")
    cache_dir: Optional[Path] = Field(None, description="Optional cache directory")
    path_resolution_strategy:  Literal['absolute', 'relative'] = 'relative'

class KernelManagerConfig(BaseModel):
    """SPICE kernel management configuration"""
    leapsecond_kernel: Path
    meta_kernel: Path
    generic_kernel_dir:  Path
    auto_download:  bool = Field(False, description="Auto-download missing kernels")
    kernel_cache_ttl: int = Field(86400, description="Cache TTL in seconds")

class GeolocationConfig(BaseModel):
    """Geolocation processing configuration"""
    instrument_name: str
    time_field: str
    reference_frame: str = "J2000"
    aberration_correction: str = "NONE"
    minimum_correlation:  Optional[float] = Field(None, description="Image matching threshold")

class PSFConfig(BaseModel):
    """Point Spread Function configuration"""
    psf_lat_sample_dist_deg: float = 0.0001
    psf_lon_sample_dist_deg: float = 0.0001

class MonteCarloConfig(BaseModel):
    """Monte Carlo analysis configuration"""
    seed: Optional[int] = None
    n_iterations: int = 100
    performance_threshold_m: float = 250.0

# note the correction-specific design has evolved beyond what is sketched here. see #125 for module details.
class CorrectionConfig(BaseModel):
    """Correction module configuration"""
    monte_carlo:  MonteCarloConfig
    psf_config: PSFConfig
    # etc
    error_threshold_m: float = 250.0

class MissionConfig(BaseModel):
    """Mission-specific configuration"""
    mission_name: str
    spacecraft_id: Optional[int] = None
    instruments: List[str] = Field(default_factory=list)
    kernel_mappings: Dict[str, str] = Field(default_factory=dict)

class CurryerConfig(BaseModel):
    """Root Curryer configuration - all modules"""
    platform: PlatformConfig
    paths: PathConfig
    kernel_manager: KernelManagerConfig
    geolocation:  GeolocationConfig
    correction: CorrectionConfig
    mission: MissionConfig

    class Config:
        env_prefix = 'CURRYER_'

# Mission-specific config example
class CLARREOConfig(CurryerConfig):
    """CLARREO mission-specific configuration"""
    def __init__(self, **kwargs):
        # Set CLARREO defaults
        if 'mission' not in kwargs:
            kwargs['mission'] = MissionConfig(
                mission_name="CLARREO",
                spacecraft_id=-125544,
                instruments=["CPRS_HYSICS"],
                kernel_mappings={
                    "constant_kernel": {"hysics": "cprs_hysics_v01.attitude. ck. json"},
                    "offset_kernel": {"az": "cprs_az_v01.attitude.ck.json"}
                }
            )
        super().__init__(**kwargs)

Usage Patterns

# From file
config = CurryerConfig.from_file('myconfig.yaml')

# Mission-specific with defaults
config = CLARREOConfig(
    platform=PlatformConfig(storage_backend='aws', data_dir='/mnt/clarreo'),
    # ... other overrides
)

# From environment variables (using pydantic-settings)
config = CurryerConfig()  # Auto-loads CURRYER_* env vars

# Programmatic
config = CurryerConfig(
    platform=PlatformConfig(data_dir="./data", storage_backend="local", parallel_workers=8),
    paths=PathConfig(kernel_dir="./kernels", output_dir="./output"),
    # ... etc
)

Team Input Needed

• How deep should the nesting/inheritance go?
• Which existing config structures to retrofit first?
• What should the user/developer config update process look like?
• How to document/validate for new missions or platforms most easily?
• Should we support both JSON and YAML config files?

Example Features/Benefits

• type-safety (find config errors at startup not runtime)
• inline docstrings (auto-docs)
• per-mission override pattern
• Config file and envvar convenience
• Could support: AWS, Local, CLARREO, etc. with small 1-line switches
• Modular with new Curryer modules/packages (Correction, Geolocation, Kernel creation...)
• Clear separation of concerns (platform vs paths vs kernels vs mission)
• more!?

This is a starting idea — let's discuss as a team what the best structure/practices would be to make Curryer maximally useful and maintainable.