feat: add LearnabilityBatchSampler — variance-weighted example selection

[LakshyAAAgrawal]

Summary

Add a new BatchSampler that prioritizes training examples where candidate performance varies most — the examples most likely to benefit from prompt improvements.

How it works

For each training example, compute the variance of scores across all evaluated candidates. High variance means candidates disagree on this example, so it's "learnable" — better prompts can make a bigger difference. Low variance (all candidates score similarly) means the example provides less optimization signal.

Parameters

• minibatch_size — examples per minibatch
• temperature — controls sampling concentration:
  • 0.0 — deterministic: always pick the top-K highest-variance examples
  • 1.0 (default) — sample proportional to variance
  • > 1.0 — flatten toward uniform (exploration)
  • < 1.0 — sharpen toward highest-variance (exploitation)
• min_candidates — minimum candidates evaluated before variance weighting activates (falls back to EpochShuffledBatchSampler until then)

Usage

from gepa.strategies.learnability_sampler import LearnabilityBatchSampler

result = gepa.optimize(
    ...,
    batch_sampler=LearnabilityBatchSampler(minibatch_size=5, temperature=0.5),
)

Test plan

• 483 tests pass (7 new)
• pyright: 0 errors
• Tests cover: variance computation, temperature=0 deterministic mode, fallback behavior, determinism with seed, high/low temperature distribution shapes

🤖 Generated with Claude Code

[semanticdiff-com]

Review changes with  

Changed Files

File	Status
src/gepa/strategies/learnability_sampler.py	0% smaller
tests/test_learnability_sampler.py	0% smaller