Skip to content

fix(harness): install declared submission packages - #8

Open
shravandoda wants to merge 1 commit into
Saivineeth147:mainfrom
shravandoda:fix/extra-pip-packages
Open

fix(harness): install declared submission packages#8
shravandoda wants to merge 1 commit into
Saivineeth147:mainfrom
shravandoda:fix/extra-pip-packages

Conversation

@shravandoda

Copy link
Copy Markdown

Summary

  • Install a submission's pinned extra_pip_packages in the Modal image.
  • Keep prefetch on the frozen base environment.
  • Install submission dependencies while building the image, before timed training begins.

Why

TASK.md permits pinned open-source packages and submissions/TEMPLATE/config.yaml
documents extra_pip_packages, but the Modal verifier currently ignores that field.
Submissions using an allowed package therefore fail before training.

Validation

  • python -m py_compile harness/modal_verify.py
  • Existing prefetch behavior remains unchanged.

@Saivineeth147

Copy link
Copy Markdown
Owner

Good catch, this field was documented but never wired up. Reviewed the diff and the approach is right, but one change needed before merge: build two images, not one. The train sandbox can use the augmented image, but eval_run has to stay on the frozen env.lock image. Otherwise a submission's pip package can patch the evaluator at import time (sitecustomize, a .pth shim, or shadowing transformers/datasets) and rig its own score. The fresh eval sandbox is our anti-tampering boundary, so nothing submission-controlled should be in it.

Concretely: keep image = build_image(modal, sub_cfg...) for train_run, and pass a separate build_image(modal) (no extras) into eval_run.

Two smaller things while you're in there: reject anything in extra_pip_packages that isn't pinned with ==, and print the installed list into the run log so it lands in the verification report.

Update this and I'll merge same day, it unblocks your #9.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants