Hands-on Discussion: Chapter 5 - Shaping model architectures via width pruning

[peremartra]

##Discussion
This is the space to share your findings from the experiments in Chapter 5. In this lab, you went beyond block-level surgery to target individual neurons inside the MLP module — choosing which ones to keep based on weight magnitude, activation patterns, or a combination of both.

The exercises in this chapter are designed to stress-test your intuition: How far can you push pruning before the model breaks? Does aligning dimensions to GPU constraints actually help on your hardware? And what happens when you deliberately keep the wrong neurons?

🎯 The Challenges

Pick one (or more) and share what you find:

1. Hardware Alignment (Tensor Cores): Modify prune_neuron_pairs (or use expansion_divisor in optipfair) so the resulting intermediate_size is divisible by 64. How many neurons does the adjustment add or subtract compared to the naive percentage? Do you measure any tokens/second improvement on your GPU?

2. Finding the Breaking Point: Push static pruning to 60% and 75%. At what threshold does the model stop generating grammatically correct text? Is the collapse gradual or does it happen suddenly?

3. Validation by Inversion (Sanity Check): Flip the selection logic — prune the most important neurons and keep the weakest ones. Compare perplexity against a correctly pruned model. Does the magnitude metric actually capture what matters?

4. Mixed-Domain Calibration: Build a calibration dataloader that mixes two contrasting datasets (e.g., 50% WikiText + 50% GSM8K, or 50% SMS + 50% TruthfulQA). Does the resulting model find a middle ground, or does it underperform both specialized versions? What happens to instruction-following metrics like IFEval?

📊 Share Your Results

Model analyzed:        [e.g., Llama-3.2-1B]
Challenge attempted:   [e.g., Finding the Breaking Point]
Pruning % / method:    [e.g., 60% static / 20% hybrid SMS+Wiki]
Key metric:            [e.g., Perplexity jumped from X to Y at 65%]
What surprised you:    [e.g., The collapse was sudden, not gradual]

Questions are just as welcome as results — if you hit an OOM error, found an unexpected benchmark behavior, or noticed something the chapter didn't predict, share it here.