Harden reward integrity across article-suite tasks - #24
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f93b103800
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| def _copy_bundle(self, destination: Path) -> Path: | ||
| source_root = self.policy_path.parent | ||
| _reject_unsafe_bundle(source_root) | ||
| bundle = destination / "bundle" | ||
| shutil.copytree(source_root, bundle) | ||
| relative_policy = self.policy_path.relative_to(source_root) |
There was a problem hiding this comment.
Preserve the Atari artifact root when isolating policy modules
For Atari overrides such as the documented policies/breakout_native_repeat_1.py, this copies only final_artifact/policies/ and then executes the module from that new directory. The Atari policy contract explicitly allows policies to load files anywhere below final_artifact/ (tasks/simulation_heuristics_atari57_v1/task_context/policy_api.md:40), so a valid nested module that reads a root-level shared table, manifest-derived data, or an adjacent artifact directory will now fail in the worker even though it worked before isolation. Copy the validated artifact root (while retaining the module's path relative to it), rather than using the module's parent as the bundle root.
Useful? React with 👍 / 👎.
Summary
agentin a Landlock worker outside the root verifier process;0on hidden-path access, web retrieval, source copying, unsafe bundles, import hooks, process escape, reward tampering, or fabricated Atari57 accounting;reward.jsonplus separateintegrity.jsonevidence;Verification
uv run pytest -q: 116 passed, 18 skippeduv run ruff check .: passuv run python scripts/validate_tasks.py: all 9 passbench tasks check --level publication-grade: all 9 pass/verifier/config.tomlnor write/logs/verifier/reward.txtCompatibility
This intentionally changes task digests and scoring trust boundaries. Existing article-suite leaderboard runs remain historical protocol-v2.2 artifacts and must not be treated as integrity-verified under this new task revision.