external validation: reproduce CML benchmark and Docker walkthrough

[safal207]

Context

CML is moving from an internal open-source artifact toward an externally reproducible research prototype.

The strongest next trust signal is simple:

Someone outside the maintainer loop can run the project and reproduce the documented results.

This issue is for contributors, reviewers, or researchers who want to validate the current CML artifact on a clean local machine.

Goal

Run the current CML validation path and submit a short external validation note.

Suggested output path:

docs/evidence/external_validation/<YYYY-MM-DD>-<github-handle>.md

What to run

1. Clone

git clone https://github.com/safal207/Causal-Memory-Layer.git
cd Causal-Memory-Layer

2. Install

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -e ".[dev]"

Windows PowerShell:

.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
pip install -e ".[dev]"

3. Run tests

pytest

4. Run benchmark

python scripts/run_safety_eval.py

5. Run Docker demo

docker compose up --build

Then follow:

docs/demo/DOCKER_CAUSAL_MEMORY_WALKTHROUGH.md

Validation note format

Please create a short note with this structure:

# External Validation Note

Validator: @your-handle
Date: YYYY-MM-DD
Repository commit: <commit-sha>

## Environment

- OS:
- Python version:
- Docker version:

## Commands run

```bash
pip install -e ".[dev]"
pytest
python scripts/run_safety_eval.py
docker compose up --build

Results

• Tests: pass/fail
• Benchmark: matched X/Y
• Docker demo: pass/fail

Notes

• What worked:
• What was confusing:
• Any reproduction issues:

Interpretation

The current CML artifact is / is not reproducible for the documented test, benchmark, and Docker demo path.

Non-claims

This validation does not prove production safety, compliance, or complete coverage.

## Acceptance criteria

- [ ] A validation note is added under `docs/evidence/external_validation/`.
- [ ] The note records OS, Python version, Docker version, and commit SHA.
- [ ] The note records `pytest` result.
- [ ] The note records benchmark result from `python scripts/run_safety_eval.py`.
- [ ] The note records whether Docker walkthrough worked.
- [ ] Any failure or confusion is reported honestly.
- [ ] No broad safety, compliance, or production-readiness claim is made.

## Non-goals

- Do not change CML audit semantics in this issue.
- Do not add new benchmark fixtures here.
- Do not require hosted services or private credentials.
- Do not claim the project is production-certified.

## Why this matters

This is a Nash-loop contribution:

```text
Contributor gets a clear, useful, low-risk task.
Project gets external reproducibility evidence.
Grant reviewers get a stronger trust signal.
Future users get a clearer install/demo path.

The target for the next grant-ready phase is:

2-5 independent external validation notes.

That would make the CML evidence package materially stronger than author-only claims.

[safal207]

Hi @marcotannoia — if you are interested, this issue could be a good fit because your earlier Docker contribution helped move CML toward a more reproducible local setup.

No pressure at all, but an external validation note from your environment would be genuinely useful for CML's reproducibility evidence.

The task is intentionally lightweight: run the documented test, benchmark, and Docker walkthrough path, then add a short note under:

docs/evidence/external_validation/

This would help show that someone outside the maintainer loop can reproduce the current artifact from public instructions.

[safal207]

2026-06-06 update: include MCP demo runner in the external validation path

CML now has a one-command local MCP demo runner merged via #122.

External validators can include this additional step after the benchmark path:

python scripts/run_mcp_demo_payloads.py

Expected sections printed:

health
audit_trace
evaluate_cause_band

Suggested updated validation command sequence:

git clone https://github.com/safal207/Causal-Memory-Layer.git
cd Causal-Memory-Layer
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -e ".[dev]"
pytest
python scripts/run_safety_eval.py
python scripts/run_mcp_demo_payloads.py
docker compose up --build

Windows PowerShell activation can still use:

.venv\Scripts\Activate.ps1

Please add the MCP demo result to the validation note:

- MCP demo runner: pass/fail
- MCP demo sections observed: health / audit_trace / evaluate_cause_band

Interpretation boundary remains unchanged: this helps validate local reproducibility of the research prototype and demo path. It does not prove production safety, enforcement, compliance readiness, or stable Cause Band semantics.