[recipe] Add McPAL — PAL's search configuration with MAC's gradient momentum - #9
Merged
Conversation
McPAL crosses two recipes already hosted here, differing from each by a single
knob: PAL's search configuration with MAC's gradient momentum.
vs pal__sitawarin2024: momentum 0 -> 0.6
vs mac__zhang2024: n_candidates 256 -> 128
PAL contributes the search budget (128 candidates from a top-256 gradient
ranking, single-token replacement, 1.1x oversampling); MAC contributes momentum
on the ranking gradient (mu=0.6).
Uses GCGPlusOptimizer since PALOptimizer has no momentum. PAL's two remaining
parameters are no-ops under a self-proxy -- proxy filtering is disabled whenever
proxy_model is model, and its unconditional retokenization is
use_retokenize=True -- both noted in the docstring.
No reproduction tag: this is a crossover of two methods, not a reproduction of
either paper.
Checks: ruff check and ty check pass on the new file, and ruff format reports it
already formatted. recipe_hub/__init__.py fails ruff format both before and
after this change, so that is pre-existing and deliberately left untouched
rather than mixed into this PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
mcpalto the Recipe Hub. It crosses two recipes already hosted here,differing from each by exactly one knob:
pal__sitawarin2024mac__zhang2024momentumn_candidatesPAL contributes the search budget — 128 candidates from a top-256 gradient
ranking, single-token replacement, 1.1× oversampling. MAC contributes momentum
on the ranking gradient,
m ← μ·m + (1−μ)·grad, μ=0.6.Uses
GCGPlusOptimizer, sincePALOptimizerhas no momentum. PAL's tworemaining parameters are no-ops under a self-proxy — proxy filtering is disabled
whenever
proxy_model is model, and its unconditional retokenization isuse_retokenize=True— both noted in the docstring.No reproduction tag, per the naming convention: this is a crossover of two
methods, not a reproduction of either paper.
Why
Momentum and candidate budget are the two axes these recipes differ on, and the
hub currently exposes them only in their published pairings — PAL without
momentum, MAC with a 256-candidate budget.
mcpalfills the remaining corner,so the two can be varied independently from a hosted starting point.
Checks
ruff check— passes on the new filety check— passes on the new fileruff format --check— the new file is already formattedtropt/recipe_hub/__init__.pyfailsruff format --checkboth before andafter this change (verified by stashing), so it is pre-existing. Left
untouched rather than mixing unrelated reformatting into this PR — happy to fix
it separately if you'd prefer.
I could not run the full
pytestsuite here (nopydanticin the sandboxinterpreter), so CI is the authority on that. The change is additive: one new
module, one registry entry, one README row.
Files
tropt/recipe_hub/McPAL.py— newtropt/recipe_hub/__init__.py— import +"mcpal"registry entrytropt/recipe_hub/README.md— table row