Skip to content

Commit d7cd3f8

Browse files
ammedina88tensorflower-gardener
authored andcommitted
Add an announcement on the public README about the new fast implementation of DP-SGD.
PiperOrigin-RevId: 512930920
1 parent a3e8a45 commit d7cd3f8

File tree

1 file changed

+28
-28
lines changed

1 file changed

+28
-28
lines changed

Diff for: README.md

+28-28
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,14 @@ issues currently open.
1111

1212
## Latest Updates
1313

14-
2020-12-21: A new
15-
[vectorized version of the TF 2 optimizer](https://github.com/tensorflow/privacy/blob/master/tensorflow_privacy/privacy/optimizers/dp_optimizer_keras_vectorized.py)
16-
is available, which can deliver much faster performance. We recommend trying it
17-
first, and to fall back to using the original non-vectorized version only if
18-
this fails. We are thankful to the
19-
[authors of this paper](https://arxiv.org/abs/2010.09063) for spurring this
20-
change.
14+
2023-02-21: A new implementation of efficient per-example gradient clipping is
15+
now available for
16+
[DP keras models](https://github.com/tensorflow/privacy/tree/master/tensorflow_privacy/privacy/keras_models)
17+
consisting only of Dense and Embedding layers. The models use the fast gradient
18+
calculation results of [this paper](https://arxiv.org/abs/1510.01799). The
19+
implementation should allow for doing DP training without any meaningful memory
20+
or runtime overhead. It also removes the need for tuning the number of
21+
microbatches as it clips the gradient with respect to each example.
2122

2223
## Setting up TensorFlow Privacy
2324

@@ -32,11 +33,11 @@ installation documentation).
3233

3334
In addition to TensorFlow and its dependencies, other prerequisites are:
3435

35-
* `scipy` >= 0.17
36+
* `scipy` >= 0.17
3637

37-
* `mpmath` (for testing)
38+
* `mpmath` (for testing)
3839

39-
* `tensorflow_datasets` (for the RNN tutorial `lm_dpsgd_tutorial.py` only)
40+
* `tensorflow_datasets` (for the RNN tutorial `lm_dpsgd_tutorial.py` only)
4041

4142
### Installing TensorFlow Privacy
4243

@@ -84,18 +85,16 @@ GitHub pull requests. To speed the code review process, we ask that:
8485
## Tutorials directory
8586

8687
To help you get started with the functionalities provided by this library, we
87-
provide a detailed walkthrough [here](tutorials/walkthrough/README.md) that
88-
will teach you how to wrap existing optimizers
89-
(e.g., SGD, Adam, ...) into their differentially private counterparts using
90-
TensorFlow (TF) Privacy. You will also learn how to tune the parameters
91-
introduced by differentially private optimization and how to
92-
measure the privacy guarantees provided using analysis tools included in TF
93-
Privacy.
94-
95-
In addition, the
96-
`tutorials/` folder comes with scripts demonstrating how to use the library
97-
features. The list of tutorials is described in the README included in the
98-
tutorials directory.
88+
provide a detailed walkthrough [here](tutorials/walkthrough/README.md) that will
89+
teach you how to wrap existing optimizers (e.g., SGD, Adam, ...) into their
90+
differentially private counterparts using TensorFlow (TF) Privacy. You will also
91+
learn how to tune the parameters introduced by differentially private
92+
optimization and how to measure the privacy guarantees provided using analysis
93+
tools included in TF Privacy.
94+
95+
In addition, the `tutorials/` folder comes with scripts demonstrating how to use
96+
the library features. The list of tutorials is described in the README included
97+
in the tutorials directory.
9998

10099
NOTE: the tutorials are maintained carefully. However, they are not considered
101100
part of the API and they can change at any time without warning. You should not
@@ -110,8 +109,8 @@ directory, but rather intended as a convenient archive.
110109

111110
## TensorFlow 2.x
112111

113-
TensorFlow Privacy now works with TensorFlow 2! You can use the new
114-
Keras-based estimators found in
112+
TensorFlow Privacy now works with TensorFlow 2! You can use the new Keras-based
113+
estimators found in
115114
`privacy/tensorflow_privacy/privacy/optimizers/dp_optimizer_keras.py`.
116115

117116
For this to work with `tf.keras.Model` and `tf.estimator.Estimator`, however,
@@ -120,16 +119,17 @@ you need to install TensorFlow 2.4 or later.
120119
## Remarks
121120

122121
The content of this repository supersedes the following existing folder in the
123-
tensorflow/models [repository](https://github.com/tensorflow/models/tree/master/research/differential_privacy)
122+
tensorflow/models
123+
[repository](https://github.com/tensorflow/models/tree/master/research/differential_privacy)
124124

125125
## Contacts
126126

127127
If you have any questions that cannot be addressed by raising an issue, feel
128128
free to contact:
129129

130-
* Galen Andrew (@galenmandrew)
131-
* Steve Chien (@schien1729)
132-
* Nicolas Papernot (@npapernot)
130+
* Galen Andrew (@galenmandrew)
131+
* Steve Chien (@schien1729)
132+
* Nicolas Papernot (@npapernot)
133133

134134
## Copyright
135135

0 commit comments

Comments
 (0)