Skip to content

Commit a5b54bf

Browse files
amrutheshtamruthesht
authored
Merge branch 'main' into person2-feature-engg
2 parents ce7511d + 7928b2a commit a5b54bf

27 files changed

Lines changed: 1466 additions & 3 deletions

‎.github/workflows/deploy.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,9 @@ concurrency:
1212
group: "${{ github.ref }}-${{ github.head_ref }}-${{ github.workflow }}"
1313
cancel-in-progress: false
1414

15+
permissions:
16+
contents: read
17+
1518
jobs:
1619
build:
1720
name: Build package

‎.github/workflows/docs.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ concurrency:
1414
jobs:
1515
docs:
1616
runs-on: ubuntu-latest
17+
permissions:
18+
contents: write
1719

1820
steps:
1921
- uses: actions/checkout@v6

‎.github/workflows/lint.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,9 @@ on:
66
pull_request:
77
branches: [ main, develop ]
88

9+
permissions:
10+
contents: read
11+
912
jobs:
1013
lint:
1114
runs-on: ubuntu-latest

‎.github/workflows/tests.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,9 @@ on:
88
schedule:
99
- cron: '0 0 * * *'
1010

11+
permissions:
12+
contents: read
13+
1114
jobs:
1215
build:
1316

‎Plans/person3-ml-workplan-2026-07-13.md‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# Person 3 Work Plan (ML Training & Evaluation)
2+
3+
**Owner:** Chenou
4+
**Date:** 2026-07-13
5+
**Branch:** person3-ml-training-eval
6+
**Target milestone:** Phase 3 complete by 2026-07-27
7+
8+
---
9+
10+
## Scope
11+
12+
Build the end-to-end ML pipeline for ConfoState:
13+
14+
1. Dataset assembly and split logic
15+
2. Baseline model training
16+
3. Evaluation + report generation
17+
4. Model artifact management / registry
18+
19+
Primary deliverables:
20+
- confostate/data/datasets.py
21+
- confostate/models/baseline.py
22+
- confostate/models/train.py
23+
- confostate/models/evaluate.py
24+
- confostate/models/registry.py
25+
- tests for datasets/model training/evaluation
26+
- trained artifacts in data/models/
27+
- evaluation report in docs/reports/
28+
29+
---
30+
31+
## Dependencies and Risk Controls
32+
33+
### Needed from Person 1 and 2
34+
- Stable annotations schema in data/annotations/leu_t_transporters.csv
35+
- Feature vectors for training rows (or at least a subset)
36+
37+
### Risk if features are delayed
38+
- Use a synthetic feature table to unblock model/training/evaluation development.
39+
- Keep synthetic schema aligned with expected real feature names.
40+
- Add a quick integration check that can switch from synthetic to real features by changing only file paths.
41+
42+
### Definition of done for integration
43+
- Train command runs successfully from raw tabular features + labels.
44+
- Evaluation command produces metrics and confusion matrix artifact.
45+
- Registry points to a reproducible model artifact and metadata.
46+
47+
---
48+
49+
## Week-by-Week Plan
50+
51+
## Week 1 (2026-07-13 to 2026-07-19): Pipeline Skeleton + Baselines
52+
53+
### Day 1: Data contract and dataset loader
54+
- Finalize expected columns:
55+
- keys: pdb_id, family, conformation
56+
- features: numeric columns only
57+
- Implement confostate/data/datasets.py:
58+
- load_dataset(annotations_csv, features_csv)
59+
- validate label and key integrity
60+
- make_split(strategy="stratified", test_size=0.2, random_state=...)
61+
62+
### Day 2: Baseline model module
63+
- Implement confostate/models/baseline.py:
64+
- logistic regression baseline
65+
- random forest baseline
66+
- SVM baseline
67+
- Add uniform interface:
68+
- get_model(name, random_state, class_weight)
69+
70+
### Day 3: Training pipeline
71+
- Implement confostate/models/train.py:
72+
- fit single model
73+
- cross-validation score summary
74+
- save artifact + metadata
75+
- Save outputs under data/models/{family}/{model_name}/
76+
77+
### Day 4: Evaluation module
78+
- Implement confostate/models/evaluate.py:
79+
- confusion matrix
80+
- per-class precision/recall/F1
81+
- macro and weighted scores
82+
- optional ROC-AUC (when applicable)
83+
- Emit markdown report to docs/reports/
84+
85+
### Day 5: Smoke tests + first run
86+
- Add tests for:
87+
- dataset split stability
88+
- model train/predict shape and class consistency
89+
- evaluation output keys and file generation
90+
- Run first end-to-end experiment with available data or synthetic fallback
91+
92+
## Week 2 (2026-07-20 to 2026-07-27): Hardening + Reporting + Handoff
93+
94+
### Day 6-7: Hyperparameter tuning and comparison
95+
- Add small grid/random search for each baseline model
96+
- Compare by macro-F1 and balanced accuracy
97+
- Select default baseline for each family
98+
99+
### Day 8: Registry and reproducibility
100+
- Implement confostate/models/registry.py:
101+
- register_model(family, model_name, artifact_path, metrics, data_version)
102+
- load_registered_model(family)
103+
- Include metadata fields:
104+
- timestamp, git commit hash, feature schema hash, random_state
105+
106+
### Day 9: Final report package
107+
- Produce docs/reports/phase3-baseline-report.md
108+
- Include:
109+
- train/test split method
110+
- model comparison table
111+
- per-state metrics
112+
- known limitations and next steps
113+
114+
### Day 10: Handoff to Person 4 and Person 5
115+
- Share:
116+
- top model artifacts
117+
- feature importance-compatible model outputs
118+
- stable prediction API contract for explainability and CLI
119+
120+
---
121+
122+
## Suggested File Interfaces
123+
124+
### confostate/data/datasets.py
125+
- load_dataset(annotations_csv: str, features_csv: str, family: str | None = None)
126+
- train_test_split_dataset(df, label_col="conformation", test_size=0.2, random_state=42)
127+
128+
### confostate/models/baseline.py
129+
- get_baseline_models(random_state=42) -> dict[str, estimator]
130+
- train_model(estimator, X_train, y_train)
131+
132+
### confostate/models/train.py
133+
- run_training(config: dict) -> dict
134+
- save_model_artifact(model, out_dir, metadata)
135+
136+
### confostate/models/evaluate.py
137+
- evaluate_model(model, X_test, y_test, labels=None) -> dict
138+
- write_evaluation_report(metrics: dict, output_path: str)
139+
140+
### confostate/models/registry.py
141+
- register_model(...)
142+
- get_registered_model(family: str)
143+
144+
---
145+
146+
## Daily Execution Checklist
147+
148+
- Confirm branch and pull latest main changes (if needed)
149+
- Implement one scoped unit of functionality
150+
- Add or update tests
151+
- Run local test subset
152+
- Commit with concise message
153+
- Post async update in team channel with blockers/dependencies
154+
155+
---
156+
157+
## Communication Cadence
158+
159+
- Tuesday standup: report progress, current blocker, next 48h plan
160+
- Async updates: at least every 2 working days in #confostate-dev
161+
- Dependency syncs:
162+
- with Person 2 on feature table schema
163+
- with Person 4 on importance/explanation-ready outputs
164+
165+
---
166+
167+
## Exit Criteria (Phase 3 Complete)
168+
169+
- End-to-end training command works on at least one family
170+
- >=2 baseline models compared and documented
171+
- Evaluation report generated with per-state metrics
172+
- Model registry points to reproducible artifacts
173+
- Person 4 receives model + outputs needed for explainability

‎Plans/person3-progress-memo-2026-07-13.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Person 3 Progress Memo
2+
3+
**Date:** 2026-07-13
4+
**Owner:** Chenou
5+
**Branch:** person3-ml-training-eval
6+
7+
## Summary
8+
9+
Implemented an initial, runnable scaffold for Person 3 (ML Training & Evaluation) so the team can start end-to-end model development before all upstream dependencies are finalized.
10+
11+
## Completed Work
12+
13+
1. Added detailed Person 3 execution plan and linked it from the 6-person team plan.
14+
2. Added dataset utilities for annotation/features merge, split generation, and X/y extraction.
15+
3. Added baseline model, training, evaluation, and registry modules under `confostate/models`.
16+
4. Added outline scripts for each Person 3 workstream task in `scripts/`.
17+
5. Updated package exports and dependencies to include model subpackages and `scikit-learn`.
18+
6. Documented script usage in `docs/USAGE.md`.
19+
20+
## New/Updated Paths
21+
22+
- Plans/person3-ml-workplan-2026-07-13.md
23+
- Plans/workplan-6person.md
24+
- confostate/data/datasets.py
25+
- confostate/models/__init__.py
26+
- confostate/models/baseline.py
27+
- confostate/models/train.py
28+
- confostate/models/evaluate.py
29+
- confostate/models/registry.py
30+
- confostate/__init__.py
31+
- confostate/data/__init__.py
32+
- scripts/p3_dataset_loader.py
33+
- scripts/p3_baseline_models.py
34+
- scripts/p3_training_pipeline.py
35+
- scripts/p3_evaluate_reporting.py
36+
- scripts/p3_model_registry.py
37+
- docs/USAGE.md
38+
- pyproject.toml
39+
40+
## Notes
41+
42+
- The scripts are intentionally outline-level and designed for iterative refinement.
43+
- Current workflow supports a synthetic feature table fallback if upstream feature extraction is delayed.
44+
- Preferred test environment for ConfoState runs:
45+
`/nfs/homes5/Projects/SLC26/chenou/openff/UGM2025/workshops/OpenFF/micromamba_root/envs/ConfoState`
46+
47+
## Immediate Next Steps
48+
49+
1. Add a minimal synthetic features CSV fixture for smoke testing.
50+
2. Run one full train/eval/registry cycle and capture report artifacts.
51+
3. Add pytest smoke tests for dataset merge and training script execution.

‎Plans/workplan-6person.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ ConfoState development split into 6 parallel work streams, each led by one team
1616

1717
- 2026-06-15 initial draft (AI generated)
1818
- 2026-06-29 annotated in group meeting
19+
- 2026-07-13 Person 3 execution plan added in Plans/person3-ml-workplan-2026-07-13.md
1920

2021
## General development notes
2122

@@ -176,6 +177,8 @@ ConfoState development split into 6 parallel work streams, each led by one team
176177

177178
**Assignee**: Chenou
178179

180+
Detailed execution plan: Plans/person3-ml-workplan-2026-07-13.md
181+
179182
### Tasks
180183

181184
1. **Build dataset loader**

‎README.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
1-
# ConfoState
1+
# ConfoState (WARNING — NON-FUNCTIONAL Work-in-progress)
2+
3+
**NOTE: This is an experimental project. It is not working. You're more than welcome to fork it and work on it on your own.**
4+
25

36
## Overview
47

‎confostate/__init__.py‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,5 +4,22 @@
44

55
from confostate.data.loader import load_annotations
66
from confostate.features import extract_features
7+
from confostate.models import (
8+
evaluate_model,
9+
get_baseline_models,
10+
get_registered_model,
11+
register_model,
12+
run_training,
13+
write_evaluation_report,
14+
)
715

8-
__all__ = ["load_annotations", "extract_features"]
16+
__all__ = [
17+
"extract_features",
18+
"evaluate_model",
19+
"get_baseline_models",
20+
"get_registered_model",
21+
"load_annotations",
22+
"register_model",
23+
"run_training",
24+
"write_evaluation_report",
25+
]

‎confostate/data/__init__.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,16 @@
11
"""Data loading utilities for ConfoState."""
2+
3+
from confostate.data.datasets import (
4+
build_xy,
5+
load_dataset,
6+
train_test_split_dataset,
7+
)
8+
from confostate.data.loader import load_annotations, load_from_input_dir
9+
10+
__all__ = [
11+
"build_xy",
12+
"load_annotations",
13+
"load_dataset",
14+
"load_from_input_dir",
15+
"train_test_split_dataset",
16+
]

0 commit comments

Comments
 (0)