-
Notifications
You must be signed in to change notification settings - Fork 0
Rerun Shanehsazzadeh notebook with ESM-2 650M #3
Copy link
Copy link
Open
Description
Rerun Shanehsazzadeh notebook with ESM-2 650M
Problem
The paper states that ESM-2 650M was used for all benchmarks, but shanehsazzadeh.ipynb currently uses ESM-2 3B:
esm_wrapper = esm2_wrapper_of_size("3B") # line 107This was flagged by an external reviewer comparing our reported correlations.
Fix
- Change
esm2_wrapper_of_size("3B")toesm2_wrapper_of_size("650M")innotebooks/dasm_paper/shanehsazzadeh.ipynb - Rerun the notebook to regenerate cached results in
_ignore/ - Update correlation values if they change
- Verify Table 2 (shane correlations) in the paper matches the new results
Notes
- Koenig notebook already uses 650M on main branch
- The 3B vs 650M comparison for Koenig is documented in PR Upgrade ESM model from 650M to 3B #2 and Table S1 of the paper
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels