fix(harness): install declared submission packages - #8
Conversation
|
Good catch, this field was documented but never wired up. Reviewed the diff and the approach is right, but one change needed before merge: build two images, not one. The train sandbox can use the augmented image, but eval_run has to stay on the frozen env.lock image. Otherwise a submission's pip package can patch the evaluator at import time (sitecustomize, a .pth shim, or shadowing transformers/datasets) and rig its own score. The fresh eval sandbox is our anti-tampering boundary, so nothing submission-controlled should be in it. Concretely: keep Two smaller things while you're in there: reject anything in extra_pip_packages that isn't pinned with ==, and print the installed list into the run log so it lands in the verification report. Update this and I'll merge same day, it unblocks your #9. |
Summary
extra_pip_packagesin the Modal image.Why
TASK.mdpermits pinned open-source packages andsubmissions/TEMPLATE/config.yamldocuments
extra_pip_packages, but the Modal verifier currently ignores that field.Submissions using an allowed package therefore fail before training.
Validation
python -m py_compile harness/modal_verify.py