[WIP] Ponytail's code cleanup

[matanbt]

Repo-wide over-engineering audit — run with the ponytail plugin's /ponytail-audit — and its fixes.

Net −409 lines of production code, +120 of new tests, −3 core dependencies. ruff, ty and pytest all clean.

Dependencies

pip install tropt was pulling pandas, scikit-learn and requests (and transitively scipy) for exactly one file: tropt/utils/refusal_dir.py, itself used by one recipe. It now reads the AdvBench CSV via datasets (already a dependency, lazily imported) and splits train/test with stdlib random. tenacity moves to the openai/google/voyage extras where it's actually used.

De-duplication

What	Before	After
compute_grad_from_tokens / compute_grad_from_embeds	~115 near-identical lines each	both delegate to _grad_wrt_leaves
gemini + voyage encoders	cloned transient-error predicates, retry logging, chunk loops	shared tropt/model/api_retry.py
_get_trigger_variations	copy-pasted in 3 optimizers	one vectorised random_single_flips in optimizer/utils
wandb + trackio _log	identical tensor-sanitising loop	BaseTracker._scalarize

openai also moves from a blanket retry to the shared transient-only policy, so a 4xx no longer burns five attempts.

Deletions

• Six loss "category" base classes with one subclass each. Losses resolve by parameter name, not by type, so these enforced nothing — their require_* flags now sit on the concrete losses. PrefillBasedLoss (4 subclasses) and TextBasedLoss stay because they carry something real.
• FlopCounterBase (one implementation), its never-called count_backward, and 65 lines of commented-out track_flops.
• InputsManager ABC — its __init__ raised and its abstract method had divergent signatures in both subclasses.
• NFlipScheduler ABC + ConstantScheduler; a scheduler is now any (step) -> int callable, so the constant case is a lambda and users can pass a plain function.
• Dead symbols: TextTrigger, BaseModel.__init__, Targets.n_templates, contains_loss_type, OpenAITokenizer._parse_ids, an identity lookup dict.

Docs and tooling

• CLAUDE.md pointed at runner/main.py and TESTING.md — neither exists in the repo. Removed those, the commented-out hydra deps, and the stale autodoc mocks. This is the first thing an outside contributor would have hit.
• Extracted the future-annotations AST injector (duplicated verbatim between build_docs.py and deploy_docs.yml) into docs/scripts/inject_annotations.py.
• build_docs.py cleanup uses rmtree's error hook instead of a chmod tree-walk. The Drive-lock retry stays — it's load-bearing on Windows.

Tests

New tests/test_optimizer_utils.py covers the two paths the end-to-end suite doesn't reach: the embedding branch of the gradient helper (PEZ / SoftPrompt flow) and both modes of random_single_flips. It includes a chain-rule identity — dL/d(onehot) == dL/d(embeds) @ Eᵀ — that pins the two gradient paths together, so a mis-wired leaf in either branch fails the build.

Reviewer notes

Two intentional behaviour changes:

1. random_single_flips draws all flips in one vectorised call, so a seeded run produces a different RNG stream than before (same distribution, so results are statistically equivalent — but exact reruns of old seeds won't match).
2. The refusal-direction train/test split no longer reproduces sklearn's exact permutation, so a different subset of prompts is sampled for the direction.

One hunk is deliberately not in this PR: the matching .github/workflows/deploy_docs.yml change, which swaps CI's inline heredoc for the extracted script. Until that lands, CI keeps its own working copy of the injector — no breakage, just the duplication left standing.

Three audit findings were investigated and dropped rather than forced:

• Replacing the landing-page expandable cards with native <details> — they're sphinx-design grid-item-card directives, so the swap means hand-rolling the card grid, a bigger change than the 30 lines of JS it removes.
• Generating CombinedLoss's seven require_* properties in a loop — BaseLoss defines them as ClassVars, so __getattr__ never fires, and the setattr alternative trades 20 lines for broken IDE autocomplete.
• Deleting one of DictTracker/JSONTracker/PrintTracker — each is a distinct output sink and they're public API on a package mid-promotion.

🤖 Generated with Claude Code

[matanbt]

[Human:] this is pending a human review; currently backlogged.