Skip to content

Commit 7101912

Browse files
committed
release: harden artifacts for v0.2.1
1 parent 6b463f9 commit 7101912

11 files changed

Lines changed: 162 additions & 15 deletions

File tree

.github/workflows/ci.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ jobs:
3434
run: |
3535
ruff format --check rewardharness src scripts examples tests vanilla
3636
ruff check rewardharness src scripts examples tests vanilla
37-
mypy rewardharness
37+
mypy rewardharness scripts/check_distribution.py scripts/check_release_metadata.py
3838
- name: Rating and shell integrity
3939
run: |
4040
python scripts/check_rating_integrity.py
@@ -93,6 +93,7 @@ jobs:
9393
run: |
9494
python -m build
9595
python -m twine check dist/*
96+
python scripts/check_distribution.py
9697
9798
- name: Test wheel installation
9899
run: |

CHANGELOG.md

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,22 @@ All notable changes to RewardHarness are recorded here. Versions follow [SemVer]
44

55
## [Unreleased]
66

7-
Future changes after v0.2.0.
7+
Future changes after v0.2.1.
8+
9+
## [0.2.1] — 2026-08-18
10+
11+
### Added
12+
13+
- Added `scripts/check_distribution.py`, a deterministic release audit that
14+
rejects stale or extra artifacts, unsafe archive paths, inconsistent package
15+
metadata, missing runtime resources, and non-runtime files leaked into the
16+
wheel.
17+
18+
### Changed
19+
20+
- Local release checks and CI now validate the exact wheel/sdist contents after
21+
every build, including the PEP 561 marker, CLI entry point, packaged Library,
22+
scoring templates, source documentation, and PyPI project URLs.
823

924
## [0.2.0] — 2026-08-17
1025

@@ -147,6 +162,7 @@ deprecated `src` namespace remains as a compatibility layer for v0.2.
147162
- `make demo` and `make benchmark` default to `--library-dir examples/seed_library` for non-empty starting state.
148163
- `make help` is now a credentials matrix showing what each target actually needs.
149164

165+
[0.2.1]: https://github.com/TIGER-AI-Lab/RewardHarness/releases/tag/v0.2.1
150166
[0.2.0]: https://github.com/TIGER-AI-Lab/RewardHarness/releases/tag/v0.2.0
151167
[0.1.2]: https://github.com/TIGER-AI-Lab/RewardHarness/releases/tag/v0.1.2
152168
[0.2.0-rc1]: https://github.com/TIGER-AI-Lab/RewardHarness/releases/tag/v0.2.0-rc1

CITATION.cff

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
cff-version: 1.2.0
22
message: "If you use this software, please cite the paper."
33
title: "RewardHarness: Self-Evolving Agentic Post-Training"
4-
version: "0.2.0"
4+
version: "0.2.1"
55
date-released: "2026-08-17"
66
type: software
77
authors:

CONTRIBUTING.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,5 +60,7 @@ Release candidates use PEP 440 versions such as `0.2.0rc1` and matching tags
6060
such as `v0.2.0-rc1`. The tag workflow publishes the same validated artifacts
6161
to PyPI and GitHub through trusted publishing. Run
6262
`rewardharness release-status` and `python scripts/check_release_metadata.py`
63-
before tagging; the workflow also installs the published PyPI artifact and
64-
verifies the resulting GitHub Release before reporting success.
63+
before tagging. `make release-check` also runs `scripts/check_distribution.py`
64+
to reject stale artifacts, metadata drift, and missing package resources; the
65+
workflow then installs the published PyPI artifact and verifies the resulting
66+
GitHub Release before reporting success.

Makefile

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ test:
3939
quality:
4040
ruff format --check rewardharness src scripts examples tests vanilla
4141
ruff check rewardharness src scripts examples tests vanilla
42-
mypy rewardharness
42+
mypy rewardharness scripts/check_distribution.py scripts/check_release_metadata.py
4343
python scripts/check_rating_integrity.py
4444
python scripts/check_migration_coverage.py
4545
python scripts/check_release_metadata.py
@@ -52,6 +52,7 @@ release-check: quality test
5252
rm -rf build dist rewardharness.egg-info
5353
python -m build
5454
python -m twine check dist/*
55+
python scripts/check_distribution.py
5556

5657
demo:
5758
rewardharness evolve \

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,8 @@ Read [`WALKTHROUGH.md`](WALKTHROUGH.md) for the 9-step path from `git clone` to
4343

4444
## Updates
4545

46+
- **2026-08-18**`v0.2.1`: deterministic wheel/sdist content auditing,
47+
stricter release metadata verification, and safer clean-build publishing.
4648
- **2026-08-17**`v0.2.0`: stable PyPI release with verified wheel/sdist,
4749
PEP 561 typing metadata, richer project links, and end-to-end trusted
4850
publishing checks.

SECURITY.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Only the current `main` branch and the most recent tagged release receive securi
2323
| Version | Supported |
2424
|---|---|
2525
| `main` ||
26-
| `v0.2.0` (latest) ||
26+
| `v0.2.1` (latest) ||
2727
| earlier ||
2828

2929
## Disclosure history

rewardharness/_version.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
"""Single source of truth for the RewardHarness package version."""
22

3-
__version__ = "0.2.0"
3+
__version__ = "0.2.1"

scripts/check_distribution.py

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
#!/usr/bin/env python3
2+
"""Validate the exact contents and metadata of release distributions."""
3+
4+
from __future__ import annotations
5+
6+
import sys
7+
import tarfile
8+
import zipfile
9+
from email.message import Message
10+
from email.parser import BytesParser
11+
from email.policy import default
12+
from pathlib import Path, PurePosixPath
13+
from typing import NoReturn
14+
15+
ROOT = Path(__file__).resolve().parent.parent
16+
DIST = ROOT / "dist"
17+
18+
19+
def _fail(message: str) -> NoReturn:
20+
raise SystemExit(f"distribution check failed: {message}")
21+
22+
23+
def _validate_paths(names: set[str], archive: str) -> None:
24+
unsafe = sorted(
25+
name
26+
for name in names
27+
if PurePosixPath(name).is_absolute() or ".." in PurePosixPath(name).parts
28+
)
29+
if unsafe:
30+
_fail(f"{archive} contains unsafe paths: {unsafe}")
31+
32+
33+
def _metadata(raw: bytes, archive: str) -> Message:
34+
message = BytesParser(policy=default).parsebytes(raw)
35+
required = {
36+
"Name": "rewardharness",
37+
"Requires-Python": ">=3.10",
38+
"License-Expression": "Apache-2.0",
39+
"Description-Content-Type": "text/markdown",
40+
}
41+
for field, expected in required.items():
42+
if message[field] != expected:
43+
_fail(f"{archive} {field} is {message[field]!r}, expected {expected!r}")
44+
classifiers = set(message.get_all("Classifier", []))
45+
if "Typing :: Typed" not in classifiers:
46+
_fail(f"{archive} does not declare inline typing support")
47+
project_urls = {value.split(",", 1)[0].strip() for value in message.get_all("Project-URL", [])}
48+
expected_urls = {"Homepage", "Documentation", "Repository", "Changelog", "Issues", "Paper"}
49+
if missing := expected_urls - project_urls:
50+
_fail(f"{archive} is missing project URLs: {sorted(missing)}")
51+
return message
52+
53+
54+
def main() -> int:
55+
sys.path.insert(0, str(ROOT))
56+
from rewardharness.release import ReleaseIdentity
57+
58+
identity = ReleaseIdentity.current()
59+
wheel = DIST / f"rewardharness-{identity.package_version}-py3-none-any.whl"
60+
sdist = DIST / f"rewardharness-{identity.package_version}.tar.gz"
61+
expected_artifacts = {wheel, sdist}
62+
actual_artifacts = (
63+
{path for path in DIST.iterdir() if path.is_file()} if DIST.is_dir() else set()
64+
)
65+
if actual_artifacts != expected_artifacts:
66+
unexpected = sorted(path.name for path in actual_artifacts - expected_artifacts)
67+
missing = sorted(path.name for path in expected_artifacts - actual_artifacts)
68+
_fail(f"artifact set mismatch; missing={missing}, unexpected={unexpected}")
69+
70+
dist_info = f"rewardharness-{identity.package_version}.dist-info"
71+
with zipfile.ZipFile(wheel) as zip_archive:
72+
wheel_names = set(zip_archive.namelist())
73+
_validate_paths(wheel_names, wheel.name)
74+
wheel_metadata = _metadata(zip_archive.read(f"{dist_info}/METADATA"), wheel.name)
75+
required_wheel = {
76+
"rewardharness/py.typed",
77+
"rewardharness/resources/library/registry.json",
78+
"rewardharness/resources/score_guidelines/template1_instruction_following.md",
79+
"rewardharness/resources/score_guidelines/template2_visual_quality.md",
80+
f"{dist_info}/entry_points.txt",
81+
}
82+
missing_wheel = required_wheel - wheel_names
83+
if missing_wheel:
84+
_fail(f"{wheel.name} is missing runtime files: {sorted(missing_wheel)}")
85+
forbidden_prefixes = ("tests/", "scripts/", "examples/", "vanilla/")
86+
leaked = sorted(name for name in wheel_names if name.startswith(forbidden_prefixes))
87+
if leaked:
88+
_fail(f"{wheel.name} contains non-runtime files: {leaked}")
89+
90+
sdist_root = f"rewardharness-{identity.package_version}"
91+
with tarfile.open(sdist, "r:gz") as tar_archive:
92+
sdist_names = {member.name for member in tar_archive.getmembers()}
93+
_validate_paths(sdist_names, sdist.name)
94+
member = tar_archive.getmember(f"{sdist_root}/PKG-INFO")
95+
extracted = tar_archive.extractfile(member)
96+
if extracted is None:
97+
_fail(f"{sdist.name} has an unreadable PKG-INFO")
98+
sdist_metadata = _metadata(extracted.read(), sdist.name)
99+
required_sdist = {
100+
f"{sdist_root}/README.md",
101+
f"{sdist_root}/CHANGELOG.md",
102+
f"{sdist_root}/CITATION.cff",
103+
f"{sdist_root}/LICENSE",
104+
f"{sdist_root}/rewardharness/py.typed",
105+
f"{sdist_root}/configs/default.yaml",
106+
}
107+
missing_sdist = required_sdist - sdist_names
108+
if missing_sdist:
109+
_fail(f"{sdist.name} is missing source files: {sorted(missing_sdist)}")
110+
111+
for message, archive_name in (
112+
(wheel_metadata, wheel.name),
113+
(sdist_metadata, sdist.name),
114+
):
115+
if message["Version"] != identity.package_version:
116+
_fail(
117+
f"{archive_name} version is {message['Version']!r}, "
118+
f"expected {identity.package_version!r}"
119+
)
120+
print(f"distribution artifacts: 2/2 valid for {identity.package_version}")
121+
return 0
122+
123+
124+
if __name__ == "__main__":
125+
raise SystemExit(main())

tests/test_infrastructure.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ def test_cli_reports_package_version(capsys):
9797
with pytest.raises(SystemExit) as raised:
9898
build_parser().parse_args(["--version"])
9999
assert raised.value.code == 0
100-
assert capsys.readouterr().out.strip() == "rewardharness 0.2.0"
100+
assert capsys.readouterr().out.strip() == "rewardharness 0.2.1"
101101

102102

103103
def test_gemini_text_and_candidate_fallback(monkeypatch):

0 commit comments

Comments
 (0)