chore: ignore local harness artifacts - #614
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7b95c4ee63
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| payload["created_at"] = datetime.now(tz=UTC).isoformat() | ||
| payload["command_count"] = len(commands) | ||
| from harness.benchmark_envelope import add_envelope | ||
| payload = add_envelope(payload, repo=repo, profile=profile, plan_hash=command_hash) |
There was a problem hiding this comment.
For a normal run, this replaces the schema-compliant evidence payload with only the new benchmark envelope before writing the file. The existing phase-2 collector still reads top-level result_code, commands, runs, and quality from each run artifact in commands/execute-phase-2-harness.sh, and harness/schemas/harness-evidence.schema.json requires those fields, so successful runs now summarize as WARN/0 and fail schema validation; files written by this version also make --replay compare against an empty prior plan. Preserve or embed the legacy payload, or migrate every consumer/schema in the same change.
AGENTS.md reference: AGENTS.md:L112-L115
Useful? React with 👍 / 👎.
| if args.dry_run: | ||
| result["result_code"] = "WARN" if not commands else "PASS" | ||
| from harness.benchmark_envelope import add_envelope | ||
| result = add_envelope(result, repo=repo, profile=profile, plan_hash=command_hash) |
There was a problem hiding this comment.
When run --dry-run is used, this assignment discards the top-level dry-run result that was just built, including result_code, plan_hash, plan, and command_count. Existing CLI callers and harness/tests/test_run_harness.py rely on those fields to inspect the discovered plan without executing it, so dry-runs now raise KeyError/lose their primary output unless the legacy fields are preserved alongside the envelope.
AGENTS.md reference: AGENTS.md:L112-L115
Useful? React with 👍 / 👎.
|
|
||
|
|
||
| def _legacy_module() -> ModuleType: | ||
| script = Path(__file__).resolve().parents[3] / "scripts" / "run-harness.py" |
There was a problem hiding this comment.
When helios-harness is installed from harness/pyproject.toml, the wheel only packages src/harness, so the sibling repository file harness/scripts/run-harness.py is not available. In that installed layout this path resolves outside the package, making the advertised console entry point fail as soon as main() or run_runner() loads the legacy module; move the runner into the package or include the script.
AGENTS.md reference: AGENTS.md:L112-L115
Useful? React with 👍 / 👎.
- harness/src/harness/benchmark_envelope.py: add envelope.v2
shape (model_family + token_pivot + warmup) for the
second-generation benchmark runs; the v1 envelope stays
readable so old runs parse.
- harness/scripts/run-harness.py: shell-quote the binary path
before passing to Popen so paths with spaces don't break the
subprocess spawn.
|
…chain-refresh (#628) Shards the unique work from 3 helios-cli branches onto current main: - feat/benchmark-provenance (#616): benchmark_envelope + harness scripts + CI - wip/20260722-helios-harness-preservation (#614): direct envelope test - feat/toolchain-refresh (#610): package.json + pnpm-lock.yaml Co-authored-by: ForgeCode <forgecode@local>
|
Closing: shard merged in #628 (benchmark-provenance + harness-preservation + toolchain-refresh). No unique work remains. |
Preserves the post-merge local artifact hygiene follow-on after the harness packet.
Validation: git diff --check.