Skip to content

Conversation

@debanganghosh08
Copy link

This PR completes the implementation suite for Issue #88 by adding an example of Delayed Preconditioners (DP²), based Research Paper, "Differentially Private Adaptive Optimization with Delayed Preconditioners".

Technical Logic:

  1. Standard private adaptive optimizers suffer from rapid noise accumulation in the second-moment estimate. This implementation introduces the alternating-phase protocol, keeping the preconditioner stale for a window of 's' steps while accumulating noised gradients.
  2. The preconditioner is then updated using the average of this window, significantly increasing the signal-to-noise ratio of the adaptive update.

✅ Correctness Verification: I have implemented an automated state-snapshot check within the script:
Phase 1 (Staleness): The script saves a snapshot at Step 4 and asserts that the preconditioner remains identical to the initial state.
Phase 2 (Update): It saves a snapshot at Step 5 and verifies that a successful update occurred following the accumulation window.
These assertions pass consistently, proving the "Snap" update logic is functioning correctly.

This is the last PR for the Split section for Issue #88 as you asked.
Kindly review and give a feedback! Thank you.

@debanganghosh08 debanganghosh08 changed the title [Example] Implemented Delayed Preconditioners for noise reduction as per Issue #88 from Paper 4 Implemented Delayed Preconditioners for noise reduction (#88) Split PR 4 Jan 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant