Add examples implementing different mechanisms from the literature

[ryan112358]

1. Fine-Tuning Large Language Models with User-Level Differential Privacy
2. It's My Data Too: Private ML for Datasets with Multi-User Training Examples
3. On Design Principles for Private Adaptive Optimizers
4. Differentially Private Adaptive Optimization with Delayed Preconditioners
   
   ...

[amyssnippet]

to crunch on these numbers until I cry binary tears

[debanganghosh08]

Hi @ryan112358 ,
I’ve just raised a PR for this issue, implementing four distinct mechanisms from the suggested literature: ULS (Charles et al. 2024), Multi-User Attribution via Greedy Bounding, Scale-then-Privatize (Ganesh et al. 2025), and DP² (Li et al. 2023).

Regarding the DP² implementation, I’ve structured the training loop to utilize a 'stale' preconditioner that updates every 's' steps using averaged private gradients.
For the Scale-then-Privatize example, I leveraged the pre_clipping_transform hook to rescale gradients into a spherical geometry before applying the _privacy cli_p.

Could you let me know if this architectural approach is specifically the manual alternating-phase protocol for DP² - does it aligns with your expectations for the library's example suite?
I’m particularly interested to know if you'd recommend further optimizing the intra-user averaging in the ULS example using a more aggressive vmap strategy, or if the current loop-based clarity is preferred for a pedagogical example.

I’m looking forward to refine these further based on your standards.
Thanks!