feat(report): add smoke evaluation workflow

Author: AlisinaDevelo

.github/workflows/ci.yml

@@ -32,3 +32,16 @@ jobs:
         run: quorabust-serve --help
       - name: Report CLI
         run: quorabust-report --help
+      - name: Smoke model report
+        run: |
+          quorabust-train \
+            --csv examples/smoke_pairs.csv \
+            --out /tmp/quorabust-smoke.pkl \
+            --eval-fraction 0 \
+            --seed 7
+          quorabust-report \
+            --model /tmp/quorabust-smoke.pkl \
+            --artifact-label quorabust-smoke.pkl \
+            --eval-csv examples/smoke_pairs.csv \
+            --out /tmp/quorabust-smoke-model-card.md
+          test -s /tmp/quorabust-smoke-model-card.md

CHANGELOG.md

@@ -6,6 +6,7 @@
 - `quorabust-report` for generating Markdown model cards from saved artifacts and
   optional labeled holdout CSVs.
 - `GET /models` serving endpoint for safe loaded-model metadata.
+- `examples/smoke_pairs.csv` and CI coverage for the train-to-report workflow.
 
 ### Fixed
 - Capped NumPy below 2.5 so Mypy can parse dependency stubs with the configured

README.md

@@ -73,7 +73,8 @@ quorabust-report \
 The report includes artifact metadata, persisted training/eval metrics, optional holdout
 metrics, and a threshold confusion matrix. Use a real held-out CSV for comparable model
 claims; the command accepts the same `question1`, `question2`, `is_duplicate` column
-contract as training.
+contract as training. See [docs/REPORTING.md](docs/REPORTING.md) for the CI smoke
+workflow and real-evaluation checklist.
 
 ### Load a saved model
 

docs/REPORTING.md

@@ -0,0 +1,50 @@
+# Model Reporting
+
+Use `quorabust-report` to turn a saved artifact into a Markdown model card. The report
+is meant for artifact review: it shows lineage, feature schema, persisted metrics, and
+optional holdout metrics with a threshold confusion matrix.
+
+## Smoke Workflow
+
+This repository includes `examples/smoke_pairs.csv` so the full train-to-report path can
+run in CI without the Kaggle dataset.
+
+```bash
+quorabust-train \
+  --csv examples/smoke_pairs.csv \
+  --out /tmp/quorabust-smoke.pkl \
+  --eval-fraction 0 \
+  --seed 7
+
+quorabust-report \
+  --model /tmp/quorabust-smoke.pkl \
+  --artifact-label quorabust-smoke.pkl \
+  --eval-csv examples/smoke_pairs.csv \
+  --out /tmp/quorabust-smoke-model-card.md
+```
+
+The smoke dataset proves the command path works. It is not a benchmark and should not be
+used for public model-quality claims.
+
+## Real Evaluation
+
+For comparable numbers, generate the report from a held-out CSV that was not used for
+training:
+
+```bash
+quorabust-train \
+  --csv data/raw/train.csv \
+  --out models/quorabust.pkl \
+  --eval-fraction 0.1 \
+  --seed 42
+
+quorabust-report \
+  --model models/quorabust.pkl \
+  --artifact-label quorabust-tfidf-v1.pkl \
+  --eval-csv data/processed/holdout.csv \
+  --out reports/quorabust-tfidf-v1.md
+```
+
+Record the dataset source, split method, command, commit SHA, and date next to any
+published result. Do not compare artifacts unless they use the same holdout split and
+threshold.

examples/smoke_pairs.csv

@@ -0,0 +1,33 @@
+question1,question2,is_duplicate
+how do i learn python fast,what is the best way to learn python,1
+best way to study golang,how can i learn go programming,1
+what is docker used for,why do developers use docker,1
+how to deploy fastapi,steps to deploy a fastapi app,1
+what is prometheus monitoring,how does prometheus collect metrics,1
+how to improve postgres queries,ways to make postgres queries faster,1
+what is a websocket,how do websockets work,1
+how to write unit tests,what is the best way to test code,1
+what is terraform,why use terraform for infrastructure,1
+how to debug latency,how can i find slow requests,1
+what is xgboost,how does xgboost classification work,1
+how to clean text data,ways to preprocess text for ml,1
+what is the capital of france,how do i install redis,0
+best pizza in rome,how does kubernetes schedule pods,0
+how to learn python,what is the weather tomorrow,0
+postgres index tuning,best guitar strings for beginners,0
+what is a websocket,how do i bake sourdough bread,0
+fastapi deployment guide,what are symptoms of flu,0
+terraform state file,how to train for a marathon,0
+prometheus metrics,where can i buy train tickets,0
+docker compose network,what is the tallest mountain,0
+unit testing in python,how to grow tomatoes indoors,0
+machine learning model card,what is the price of gold,0
+text preprocessing pipeline,how to learn chess openings,0
+how do i cache api responses,ways to cache http api results,1
+what is rate limiting,how does api rate limiting work,1
+how to rotate logs,steps for application log rotation,1
+what is a health check,why services expose health endpoints,1
+how to tune xgboost,which xgboost parameters matter,1
+what is model drift,how do ml models drift over time,1
+redis cache invalidation,how to repair a bicycle chain,0
+openapi schema docs,what causes ocean tides,0

src/quorabust/report.py

@@ -217,6 +217,11 @@ def main(argv: list[str] | None = None) -> int:
         default=0.5,
         help="Decision threshold for the optional confusion matrix",
     )
+    parser.add_argument(
+        "--artifact-label",
+        default=None,
+        help="Public artifact label to print instead of the local model path",
+    )
     parser.add_argument("--out", type=Path, default=None, help="Write Markdown here")
     args = parser.parse_args(argv)
 
@@ -245,7 +250,7 @@ def main(argv: list[str] | None = None) -> int:
             return 1
 
     card = render_model_card(
-        artifact=str(args.model.resolve()),
+        artifact=args.artifact_label or str(args.model.resolve()),
         meta=meta,
         holdout_metrics=holdout_metrics,
     )

tests/test_report.py

@@ -61,9 +61,24 @@ def test_report_cli_writes_model_card(tmp_path):
     _df().to_csv(eval_csv, index=False)
     out = tmp_path / "MODEL_CARD.md"
 
-    assert main(["--model", str(model), "--eval-csv", str(eval_csv), "--out", str(out)]) == 0
+    assert (
+        main(
+            [
+                "--model",
+                str(model),
+                "--artifact-label",
+                "smoke-model.pkl",
+                "--eval-csv",
+                str(eval_csv),
+                "--out",
+                str(out),
+            ]
+        )
+        == 0
+    )
 
     card = out.read_text(encoding="utf-8")
+    assert "| artifact | smoke-model.pkl |" in card
     assert "## Holdout Evaluation" in card
     assert "## Confusion Matrix" in card