Rdp for subsampled gaussian mechanism under the sampling without replacement scheme #67
Conversation
…eplacement. Also, adding references for the existing Poisson sampling scheme calculations.
…dp_for_subsampled_gaussian automerged
|
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed (or fixed any issues), please reply here with What to do if you already signed the CLAIndividual signers
Corporate signers
ℹ️ Googlers: Go here for more info. |
|
@googlebot I signed it!
… On Aug 14, 2019, at 4:59 PM, googlebot ***@***.***> wrote:
@googlebot I signed it!
|
|
CLAs look good, thanks! ℹ️ Googlers: Go here for more info. |
1 similar comment
|
CLAs look good, thanks! ℹ️ Googlers: Go here for more info. |
| Accountant." AISTATS'2019. | ||
|
|
||
| A strengthened version -- Theorem 27 -- applies subsampled-Gaussian mechanism. An implementation | ||
| is available at https://github.com/yuxiangw/autodp |
There was a problem hiding this comment.
Would you be able to also include an implementation of the strengthened version? Initially, I thought we would only need the result in Theorem 27 because TensorFlow Privacy only uses the Gaussian Mechanism. But now that you have an implementation of Theorem 9, it'd be useful to keep ti for the future (when we start including other mechanisms).
There was a problem hiding this comment.
The implemented version is for the Gaussian mechanism. Although it is easy to modify it and make it more modular to cover other mechanisms. The strengthened version (Theorem 27) is somewhat tricky to implement... and the improvement over Theorem 9 is only in the third term of the expansion onwards. I think it makes sense to first merge this PR and then start another branch to investigate how to add Theorem 27 in the most light-weight manner.
|
|
||
| for i in range(2, alpha+1): | ||
| if i == 2: | ||
| log_coef_i = math.log(special.binom(alpha, i)) + i * math.log(q) |
There was a problem hiding this comment.
Do you think the use of numpy's log1p would lead to more stable calculations? Any other tricks to stabilize the calculations?
There was a problem hiding this comment.
I was following the implementation from compute_rdp. For small alpha, this should OK. For large alpha, maybe doing everything in the log-space might be better. It really depends on how special.binom is implemented in scipy.
|
|
||
| # A simple test script to demonstrate the calculated RDP | ||
|
|
||
| q= 0.01 |
| results2 = compute_rdp_sample_without_replacement(q, noise_multiplier, steps, orders) | ||
|
|
||
|
|
||
| import matplotlib.pyplot as plt |
There was a problem hiding this comment.
I believe pylint will complain about this. Perhaps best to do it at the beginning.
|
@kairouzp what is the status on this? |
|
+1 🤔 |
Function
compute_rdp_sample_without_replacementimplements Theorem 9 of Wang, Balle, Kasiviswanathan. "Subsampled Renyi Differential Privacy and Analytical Moments Accountant." AISTATS'2019.Added a test script.
Added a reference to why
compute_rdpis correctly calculating the RDP for subsampled Gaussian mechanism under the Poisson sampling scheme (independently include a data point with probability q). At least for integer order...