New FastCSP release: update to docs/example/configs/correctors

src/fairchem/applications/fastcsp/README.md

@@ -1,48 +1,39 @@
 # FastCSP: Accelerated Molecular Crystal Structure Prediction with Universal Model for Atoms
 
-FastCSP is a complete computational workflow for predicting molecular crystal structures from selected conformers by combining random structure generation and machine learning-based optimization without requiring high-accuracy DFT validation.
+FastCSP is a complete computational workflow for predicting molecular crystal structures from molecular SMILES strings by combining conformer generation, random structure generation, and machine learning-based optimization without requiring any final DFT reranking.
 
 ## Overview
 
-FastCSP provides an end-to-end workflow for crystal structure prediction that scales from initial structure generation to experimental validation:
-
 <div align="center">
 <img src="fastcsp.svg" alt="FastCSP Workflow Overview" width="800"/>
 </div>
 
 ### Workflow Stages
 
+FastCSP splits into an **upstream conformer helper** (`fastcsp-confgen`,
+SMILES → 3D geometries) and the **main 7-stage prediction workflow**
+(`fastcsp`). The two are separate CLIs: `fastcsp-confgen` writes per-molecule XYZ conformers, and `fastcsp` consumes them via the `conformers_path` column of `molecules.csv`. You can skip `fastcsp-confgen` entirely if you already have 3D
+conformers from another source.
+
+0. **Conformer Generation** (Optional, upstream — [`fastcsp-confgen`](confgen/README.md)):
+   Per molecule, generate a diverse RDKit ETKDG pool, MLIP-relax with a
+   FAIR-Chem UMA calculator, dedup on best-RMSD (Butina) + energy gate,
+   and write per-conformer geometry files that feed into Stage 1 below.
 1. **Structure Generation**: [`Genarris 3.0`](https://github.com/Yi5817/Genarris) generates putative crystal structures.
-2. **Deduplication**: Pymatgen's StructureMatcher deduplicates generated structures.
-3. **ML Relaxation**: Structures are fully relaxed using the Universal Model for Atoms (UMA) from [`fairchem`](https://fair-chem.github.io/).
-4. **Filtering & Deduplication**: Property filtering, structure deduplication using pymatgen's StructureMatcher and validation generates the energy landscape at 0 K.
-5. **Experimental Validation** (Optional): Evaluation through comparison against experimental crystal structures using PackingSimilarity from CSD Python API [requires [CCDC license](https://downloads.ccdc.cam.ac.uk/documentation/API/installation_notes.html)] or pymatgen's StructureMatcher.
-6. **Free Energy Calculations** (Optional): Free energy corrections are computed and a corrected energy landscape becomes available.
+2. **Process Generated Structures**: Pymatgen's StructureMatcher deduplicates generated structures.
+3. **MLIP Relaxation**: Structures are fully relaxed using the Universal Model for Atoms (UMA) from [`fairchem`](https://fair-chem.github.io/).
+4. **Conformer Energy Corrections** (Optional): Each relaxed crystal is split into its ``z``molecules and re-scored with a second ("corrector") UMA calculator.
+5. **Filtering**: Property filtering and structure deduplication using pymatgen's StructureMatcher. This generates the energy landscape at 0 K.
+6. **Experimental Validation** (Optional): Evaluation through comparison against experimental crystal structures using PackingSimilarity from CSD Python API [requires [CCDC license](https://downloads.ccdc.cam.ac.uk/documentation/API/installation_notes.html)] or pymatgen's StructureMatcher.
+7. **Free Energy Calculations** (Optional): Quasi-harmonic vibrational thermodynamics is computed per structure with the same UMA calculator (Gibbs free energies, entropies, and optional phonon DOS), producing a temperature-dependent free-energy landscape.
 
 ### Key Features
 
-**High-Performance Computing Integration:**
 - Native SLURM support for parallel processing across compute clusters
-- Automatic job dependency management and fault tolerance
 - Scalable from single molecules to large datasets
-
-**Advanced Structure Analysis:**
-- Multi-level structure comparison: pre/post-relaxation deduplication
-- Configurable similarity metrics for crystallographic matching
-
-**Flexible Workflow Control:**
+- Control knobs for each stage runtime/stringency tradeoff
 - Modular stage-based execution - run complete pipeline or individual steps
 - Resume capability - skip already completed stages
-- Comprehensive logging system with restart capability
-
-### Performance & Scalability
-
-FastCSP is designed for production-scale crystal structure prediction campaigns:
-- **Structure Generation**: 500+ structures per molecule within hour (depending on space group and Z complexity)
-- **ML Relaxation**: tens of seconds per structure on modern GPUs
-- **Structure Comparison**: Efficient parallel processing of large structure databases
-- **Memory Usage**: Optimized for large datasets with batching
-- **Storage**: Compressed Parquet format for efficient structure storage
 
 ## Output Directory Structure
 
@@ -77,26 +68,39 @@ your_project_root/
 └── relaxed/                        # Stage 3+: ML relaxation and analysis results
     └── uma_sm_1p1_omc_bfgs_0.01_1000_relaxcell/  # Named by ML model + optimizer settings
         ├── raw_structures/         # Stage 3: ML-relaxed crystal structures
-        │   ├── MOLECULE1/
-        │   │   └── CONFORMER1/
+        │   ├── MOLECULE1/          #   Stage 4 (compute_conformer_corrections) rewrites
+        │   │   └── CONFORMER1/     #   these parquets in place by default, adding an
+        │   │       └── partition_id=*/  # energy_corrected column (see raw_conformer_
+        │   │           └── *.parquet    # corrected_structures/ below for separate-output mode).
+        │   └── MOLECULE2/
+        │
+        ├── raw_conformer_corrected_structures/   # Stage 4 (optional): only when
+        │   ├── MOLECULE1/                        # conformer_corrections.separate_output=true
+        │   │   └── CONFORMER1/                   # (default is to rewrite raw_structures/ in place)
         │   │       └── partition_id=*/
-        │   │           └── *.parquet  # Relaxed structures with energies
+        │   │           └── *.parquet
         │   └── MOLECULE2/
         │
-        ├── filtered_structures/    # Stage 4: Energy-filtered and deduplicated structures
+        ├── filtered_structures/    # Stage 5: Energy-filtered and deduplicated structures
         │   ├── MOLECULE1.parquet   # One parquet per molecule
         │   └── MOLECULE2.parquet
         │
-        └── matched_structures/         # Stage 5 (eval): name depends on method
-            │                               #   csd      → matched_structures_csd/
-            │                               #   pymatgen → matched_structures_pmg_l<ltol>_s<stol>_a<angle_tol>/
-            ├── MOLECULE1.parquet           # Per-molecule structures with experimental similarity scores
-            └── MOLECULE2.parquet
+        ├── matched_structures/         # Stage 6 (eval): name depends on method
+        │   │                               #   csd      → matched_structures_csd/
+        │   │                               #   pymatgen → matched_structures_pmg_l<ltol>_s<stol>_a<angle_tol>/
+        │   ├── MOLECULE1.parquet           # Per-molecule structures with experimental similarity scores
+        │   └── MOLECULE2.parquet
+        │
+        └── free_energy/             # Stage 7 (optional, compute_free_energy):
+            ├── MOLECULE1.parquet    # per-structure vibrational thermo (F(T), S(T),
+            └── MOLECULE2.parquet    # optional DOS) joined with the input columns
 ```
 
 ### Key Data Files
 
-- **Parquet Files**: Compressed columnar storage containing structure data, energies, lattice parameters, and metadata
+- **Parquet Files**: Compressed columnar storage containing structure data, energies, lattice parameters, and metadata. Beyond the base ``energy_relaxed`` and ``density_relaxed`` columns, downstream stages add:
+  - ``energy_corrected``, ``energy_corrected_per_molecule``, ``correction.*``, ``validity.conformer_corrections.applied`` (Stage 4 `compute_conformer_corrections`).
+  - vibrational thermo columns (Helmholtz/Gibbs free energies, entropies, and optional phonon DOS) on a temperature grid (Stage 7 `compute_free_energy`).
 - **CIF Strings**: Stored within Parquet files for easy structure visualization and analysis
 - **JSON Files**: Raw Genarris outputs with structure information
 - **Log Files**: Comprehensive workflow logs with timestamps, stage progress, and error tracking
@@ -154,6 +158,15 @@ IHEPUG,IHEPUG_mol.xyz,"IHEPUG02,IHEPUG"
 - **(Required)** [`Genarris 3.0`](https://github.com/Yi5817/Genarris): Crystal structure generation engine
 - **(Optional)** [`CSD Python API`](https://downloads.ccdc.cam.ac.uk/documentation/API/installation_notes.html): For experimental structure comparison (requires license)
 
+### End-to-end example
+
+See [`example/`](example/) for a copy-paste-ready SMILES → predicted
+crystal-structure landscape pipeline for aspirin and glycine. It uses
+`fastcsp-confgen` to generate starting conformers, then all 7 workflow
+stages (generate → process_generated → relax → compute_conformer_corrections
+→ filter → evaluate → compute_free_energy). Edit the placeholders
+(`<PROJECT_ROOT>`, `<PATH_TO_GENARRIS>`, `<YOUR_PARTITION>`, …) and run.
+
 ### Basic Usage
 
 **Complete Workflow:**
@@ -182,14 +195,20 @@ fastcsp --config config.yaml --stages generate process_generated relax filter
 
 ### Available Workflow Stages
 
-| Stage | Description | Output |
-|-------|-------------|--------|
-| `generate` | Generate crystal structures using Genarris | `generated_structures/` |
-| `process_generated` | Process and deduplicate Genarris outputs | `raw_structures/` |
-| `relax` | Perform UMA-based structure relaxation | `relaxed/<run_name>/raw_structures/` |
-| `filter` | Property filtering and duplicate removal | `relaxed/<run_name>/filtered_structures/` |
-| `evaluate` | Compare against experimental data | `relaxed/<run_name>/matched_structures_{csd,pmg_*}/` |
-| `compute_free_energy` *(optional)* | Quasi-harmonic vibrational free energies | `relaxed/<run_name>/free_energy/` |
+The `fastcsp` CLI orchestrates the 7 stages below. `fastcsp-confgen` is a
+separate upstream CLI (see [`confgen/README.md`](confgen/README.md)) whose
+outputs feed Stage 1 via the `conformers_path` column of `molecules.csv`.
+
+| CLI | Stage | Description | Output |
+|-----|-------|-------------|--------|
+| `fastcsp-confgen` | *upstream (optional)* | SMILES → RDKit ETKDG pool → UMA relax → Butina dedup → per-conformer `.xyz` files | `<PROJECT_ROOT>/conformers/conformers_fastcsp/<name>/*.xyz` |
+| `fastcsp` | `generate` | Generate crystal structures using Genarris | `generated_structures/` |
+| `fastcsp` | `process_generated` | Process and deduplicate Genarris outputs | `raw_structures/` |
+| `fastcsp` | `relax` | Perform UMA-based structure relaxation | `relaxed/<run_name>/raw_structures/` |
+| `fastcsp` | `compute_conformer_corrections` *(optional)* | Per-molecule fragment energy corrections on relaxed parquets | `relaxed/<run_name>/raw_structures/` (in-place, or `raw_conformer_corrected_structures/`) |
+| `fastcsp` | `filter` | Property filtering and duplicate removal | `relaxed/<run_name>/filtered_structures/` |
+| `fastcsp` | `evaluate` | Compare against experimental data | `relaxed/<run_name>/matched_structures_{csd,pmg_*}/` |
+| `fastcsp` | `compute_free_energy` *(optional)* | Quasi-harmonic vibrational free energies | `relaxed/<run_name>/free_energy/` |
 
 ### Configuration
 
@@ -212,6 +231,13 @@ FastCSP uses YAML configuration files to control all workflow parameters. Exampl
 - `relax`: ML relaxation settings
   (`calculator`, `optimizer`, `fmax`, `max_steps`, `fix_symmetry`,
   `relax_cell`, `write_traj`, `traj_interval`) and SLURM block
+- `conformer_corrections`: Per-conformer fragment energy corrections (run
+  with `--stages compute_conformer_corrections`). Keys: `corrector_calculator`
+  (required), `original_calculator` (defaults to `relax.calculator`),
+  `separate_output` (default `false` → rewrite parquets in place), and a
+  `slurm` block. Adds columns `energy_corrected`,
+  `energy_corrected_per_molecule`, `correction.*`, and
+  `validity.conformer_corrections.applied` to the relaxed parquets.
 - `post_relaxation_filter`: Property cutoffs and deduplication
   (`remove_problematic`, `energy_cutoff`, `density_min_cutoff`,
   `density_max_cutoff`, `assign_groups`, `remove_duplicates`,
@@ -223,8 +249,10 @@ FastCSP uses YAML configuration files to control all workflow parameters. Exampl
   `csd.{num_cpus, python_cmd, target_rows_per_chunk, chunk_timeout}`,
   `pymatgen.{match_params, slurm}`)
 - `free_energy`: Vibrational free energy corrections (run with
-  `--stages compute_free_energy`). Keys: `calculator`, `quasiharmonic`,
-  `atom_disp`, `min_lengths`, `t_min`/`t_max`/`t_step`, `match_only`,
+  `--stages compute_free_energy`). Keys: `calculator`, `input_directory`
+  (default `filtered_structures`), `quasiharmonic`, `atom_disp`,
+  `min_lengths`, `t_min`/`t_max`/`t_step`, `match_only` (default `false`;
+  requires `input_directory: matched_structures` when `true`),
   `energy_cutoff`, `max_structures`, `structures_per_job`, `compute_dos`,
   and a `slurm` block.
 - `logging`: Log file settings (`level`, `console`)