Introducing TensorFlow Privacy: Learning with Differential Privacy for Training Data
<html> <div class=„section-divider“><hr class=„section-divider“/></div><div class=„section-content“><div class=„section-inner sectionLayout–insetColumn“><h1 name=„ac52“ id=„ac52“ class=„graf graf–h3 graf–leading graf–title“><em class=„markup–em markup–h3-em“>Introducing TensorFlow Privacy: Learning with Differential Privacy for Training Data</em></h1><div class=„uiScale uiScale-ui–regular uiScale-caption–regular u-flexCenter u-marginVertical24 u-fontSize15 js-postMetaLockup“><div class=„u-flex0“><a class=„link u-baseColor–link avatar“ href=„https://medium.com/@tensorflow?source=post_header_lockup“ data-action=„show-user-card“ data-action-source=„post_header_lockup“ data-action-value=„b1d410cb9700“ data-action-type=„hover“ data-user-id=„b1d410cb9700“ dir=„auto“><div class=„u-relative u-inlineBlock u-flex0“><img src=„https://cdn-images-1.medium.com/fit/c/100/100/1*iDQvKoz7gGHc6YXqvqWWZQ.png“ class=„avatar-image u-size50x50“ alt=„Go to the profile of TensorFlow“/><div class=„avatar-halo u-absolute u-textColorGreenNormal svgIcon“ style=„width: calc(100% + 10px); height: calc(100% + 10px); top:-5px; left:-5px“><svg viewbox=„0 0 70 70“ xmlns=„http://www.w3.org/2000/svg“><path d=„M5.53538374,19.9430227 C11.180401,8.78497536 22.6271155,1.6 35.3571429,1.6 C48.0871702,1.6 59.5338847,8.78497536 65.178902,19.9430227 L66.2496695,19.401306 C60.4023065,7.84329843 48.5440457,0.4 35.3571429,0.4 C22.17024,0.4 10.3119792,7.84329843 4.46461626,19.401306 L5.53538374,19.9430227 Z“/><path d=„M65.178902,49.9077131 C59.5338847,61.0657604 48.0871702,68.2507358 35.3571429,68.2507358 C22.6271155,68.2507358 11.180401,61.0657604 5.53538374,49.9077131 L4.46461626,50.4494298 C10.3119792,62.0074373 22.17024,69.4507358 35.3571429,69.4507358 C48.5440457,69.4507358 60.4023065,62.0074373 66.2496695,50.4494298 L65.178902,49.9077131 Z“/></svg></div></div></a></div><div class=„u-flex1 u-paddingLeft15 u-overflowHidden“><div class=„u-paddingBottom3“><a class=„ds-link ds-link–styleSubtle ui-captionStrong u-inlineBlock link link–darken link–darker“ href=„https://medium.com/@tensorflow“ data-action=„show-user-card“ data-action-value=„b1d410cb9700“ data-action-type=„hover“ data-user-id=„b1d410cb9700“ dir=„auto“>TensorFlow</a><button class=„button button–smallest u-noUserSelect button–withChrome u-baseColor–buttonNormal button–withHover button–unblock js-unblockButton u-marginLeft10 u-xs-hide“ data-action=„sign-up-prompt“ data-sign-in-action=„toggle-block-user“ data-requires-token=„true“ data-redirect=„https://medium.com/tensorflow/introducing-tensorflow-privacy-learning-with-differential-privacy-for-training-data-b143c5e801b6“ data-action-source=„post_header_lockup“>BlockedUnblock</button><button class=„button button–primary button–smallest button–dark u-noUserSelect button–withChrome u-accentColor–buttonDark button–follow js-followButton u-marginLeft10 u-xs-hide“ data-action=„sign-up-prompt“ data-sign-in-action=„toggle-subscribe-user“ data-requires-token=„true“ data-redirect=„https://medium.com/_/subscribe/user/b1d410cb9700“ data-action-source=„post_header_lockup-b1d410cb9700————————-follow_byline“>FollowFollowing</button></div><div class=„ui-caption u-noWrapWithEllipsis js-testPostMetaInlineSupplemental“><time datetime=„2019-03-06T16:01:01.530Z“>Mar 6</time></div></div></div><p name=„9688“ id=„9688“ class=„graf graf–p graf-after–h3“><em class=„markup–em markup–p-em“>Posted by </em><a href=„https://twitter.com/schmilblick42“ data-href=„https://twitter.com/schmilblick42“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“><em class=„markup–em markup–p-em“>Carey Radebaugh</em></a><em class=„markup–em markup–p-em“> (Product Manager) and </em><a href=„https://ai.google/research/people/ulfar“ data-href=„https://ai.google/research/people/ulfar“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“><em class=„markup–em markup–p-em“>Ulfar Erlingsson</em></a><em class=„markup–em markup–p-em“> (Research Scientist)</em></p><p name=„85c1“ id=„85c1“ class=„graf graf–p graf-after–p“>Today, we’re excited to announce TensorFlow Privacy (<a href=„https://github.com/tensorflow/privacy“ data-href=„https://github.com/tensorflow/privacy“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>GitHub</a>), an open source library that makes it easier not only for developers to train machine-learning models with privacy, but also for researchers to advance the state of the art in machine learning with strong privacy guarantees.</p><p name=„018f“ id=„018f“ class=„graf graf–p graf-after–p“>Modern machine learning is increasingly applied to create amazing new technologies and user experiences, many of which involve training machines to learn responsibly from sensitive data, such as personal photos or email. Ideally, the parameters of trained machine-learning models should encode general patterns rather than facts about specific training examples. To ensure this, and to give strong privacy guarantees when the training data is sensitive, it is possible to use techniques based on the theory of <a href=„https://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf“ data-href=„https://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“><em class=„markup–em markup–p-em“>differential privacy</em></a>. In particular, when training on users’ data, those techniques offer strong mathematical guarantees that models do not learn or remember the details about any specific user. Especially for deep learning, the additional guarantees can usefully strengthen the protections offered by other privacy techniques, whether established ones, such as thresholding and data elision, or new ones, like <a href=„https://www.tensorflow.org/federated“ data-href=„https://www.tensorflow.org/federated“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>TensorFlow Federated</a> learning.</p><figure name=„ac54“ id=„ac54“ class=„graf graf–figure graf–layoutOutsetLeft graf-after–p“><div class=„aspectRatioPlaceholder is-locked“ style=„max-width: 269px; max-height: 276px;“><div class=„aspectRatioPlaceholder-fill“ style=„padding-bottom: 102.60000000000001%;“/><img class=„graf-image“ data-image-id=„1*pIvYGeZiBrh0L5motrVSlw.png“ data-width=„269“ data-height=„276“ src=„https://cdn-images-1.medium.com/max/1200/1*pIvYGeZiBrh0L5motrVSlw.png“/></div></figure><p name=„673b“ id=„673b“ class=„graf graf–p graf-after–figure“>For several years, Google has spearheaded both foundational research on differential privacy as well as the development of practical differential-privacy mechanisms (see for example <a href=„https://arxiv.org/abs/1702.07476“ data-href=„https://arxiv.org/abs/1702.07476“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>here</a> and <a href=„https://ai.google/research/pubs/pub42852“ data-href=„https://ai.google/research/pubs/pub42852“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>here</a>), with a recent focus on machine learning applications (see <a href=„https://arxiv.org/abs/1607.00133“ data-href=„https://arxiv.org/abs/1607.00133“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>this</a>, <a href=„https://arxiv.org/abs/1802.08908“ data-href=„https://arxiv.org/abs/1802.08908“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>that</a>, or <a href=„https://arxiv.org/abs/1710.06963“ data-href=„https://arxiv.org/abs/1710.06963“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>this</a> research paper). Last year, Google published its <a href=„https://ai.google/education/responsible-ai-practices“ data-href=„https://ai.google/education/responsible-ai-practices“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>Responsible AI Practices</a>, detailing our recommended practices for the responsible development of machine learning systems and products; even before this publication, we have been working hard to make it easy for external developers to apply such practices in their own products.</p><p name=„5b0f“ id=„5b0f“ class=„graf graf–p graf-after–p“>One result of our efforts is today’s announcement of <a href=„https://github.com/tensorflow/privacy“ data-href=„https://github.com/tensorflow/privacy“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>TensorFlow Privacy</a> and the updated <a href=„https://arxiv.org/abs/1812.06210“ data-href=„https://arxiv.org/abs/1812.06210“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>technical whitepaper</a> describing its privacy mechanisms in more detail.</p><p name=„de9d“ id=„de9d“ class=„graf graf–p graf-after–p“>To use TensorFlow Privacy, no expertise in privacy or its underlying mathematics should be required: those using standard TensorFlow mechanisms should not have to change their model architectures, training procedures, or processes. Instead, to train models that protect privacy for their training data, it is often sufficient for you to make some simple code changes and tune the hyperparameters relevant to privacy.</p><h3 name=„3e83“ id=„3e83“ class=„graf graf–h3 graf-after–p“>An example: learning a language with privacy</h3><p name=„493a“ id=„493a“ class=„graf graf–p graf-after–h3“>As a concrete example of differentially-private training, let us consider the training of character-level, recurrent language models on text sequences. Language modeling using neural networks is an essential deep learning task, used in innumerable applications, many of which are based on <a href=„https://ai.googleblog.com/2018/05/smart-compose-using-neural-networks-to.html“ data-href=„https://ai.googleblog.com/2018/05/smart-compose-using-neural-networks-to.html“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>training with sensitive data</a>. We train two models — one in the standard manner and one with differential privacy — using the same model architecture, based on example code from the TensorFlow Privacy <a href=„https://github.com/tensorflow/privacy“ data-href=„https://github.com/tensorflow/privacy“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>GitHub repository</a>.</p><p name=„1e63“ id=„1e63“ class=„graf graf–p graf-after–p“>Both of the models do well on modeling the English language in financial news articles from the standard <a href=„https://catalog.ldc.upenn.edu/LDC99T42“ data-href=„https://catalog.ldc.upenn.edu/LDC99T42“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>Penn Treebank training dataset</a>. However, if the slight differences between the two models were due to a failure to capture some essential, core aspects of the language distribution, this would cast doubt on the utility of the differentially-private model. (On the other hand, the private model’s utility might still be fine, even if it failed to capture some esoteric, unique details in the training data.)</p><p name=„1d7d“ id=„1d7d“ class=„graf graf–p graf-after–p“>To confirm the utility of the private model, we can look at the two models’ performance on the corpus of training and test data and examine the set of sentences on which they agree and disagree. To look at their commonality, we can measure their similarity on modeled sentences to see if both models accept the same core language; in this case, both models accept and score highly (i.e., have low perplexity for) over 98% of the training data sequences. For example, both models score highly the following financial news sentences (shown in italics, as they are clearly in the distribution we wish to learn):</p><blockquote name=„6f2c“ id=„6f2c“ class=„graf graf–blockquote graf-after–p“>there was little turnover and nothing to stimulate the market</blockquote><blockquote name=„1b0c“ id=„1b0c“ class=„graf graf–blockquote graf-after–blockquote“>south korea and japan continue to be profitable</blockquote><blockquote name=„2c73“ id=„2c73“ class=„graf graf–blockquote graf-after–blockquote“>merchant banks were stronger across the board</blockquote><p name=„f06a“ id=„f06a“ class=„graf graf–p graf-after–blockquote“>To look at their differences, we can examine training-data sentences on which the two models’ scores diverge greatly. For example, all of the following three training-data sentences are scored highly and accepted by the regular language model, since they are effectively memorized during standard training. However, the differentially-private model scores these sentences very low and does not accept them. (Below, the sentences are shown in bold, because they seem outside the language distribution we wish to learn.)</p><blockquote name=„3535“ id=„3535“ class=„graf graf–blockquote graf-after–p“><strong class=„markup–strong markup–blockquote-strong“><em class=„markup–em markup–blockquote-em“>aer banknote berlitz calloway … ssangyong swapo wachter</em></strong></blockquote><blockquote name=„93bd“ id=„93bd“ class=„graf graf–blockquote graf-after–blockquote“><strong class=„markup–strong markup–blockquote-strong“><em class=„markup–em markup–blockquote-em“>the naczelnik stands too</em></strong></blockquote><blockquote name=„a364“ id=„a364“ class=„graf graf–blockquote graf-after–blockquote“><strong class=„markup–strong markup–blockquote-strong“><em class=„markup–em markup–blockquote-em“>my god and i know i am correct and innocent</em></strong></blockquote><p name=„e7f4“ id=„e7f4“ class=„graf graf–p graf-after–blockquote“>All of the above sentences seem like they should be very uncommon in financial news; furthermore, they seem sensible candidates for privacy protection, e.g., since such rare, strange-looking sentences might identify or reveal information about individuals in models trained on sensitive data. The first of the three sentences is a long sequence of random words that occurs in the training data for technical reasons; the second sentence is part Polish; the third sentence — although natural-looking English — is not from the language of financial news being modeled. These examples are selected by hand, but full inspection confirms that the training-data sentences not accepted by the differentially-private model generally lie outside the normal language distribution of financial news articles. Furthermore, by evaluating test data, we can verify that such esoteric sentences are a basis for the loss in quality between the private and the non-private models (1.13 vs. 1.19 perplexity). Therefore, although the nominal perplexity loss is around 6%, the private model’s performance may hardly be reduced at all on sentences we care about.</p><p name=„44e6“ id=„44e6“ class=„graf graf–p graf-after–p“>Clearly, at least in part, the two models’ differences result from the private model failing to memorize rare sequences that are abnormal to the training data. We can quantify this effect by leveraging our <a href=„https://arxiv.org/abs/1802.08232“ data-href=„https://arxiv.org/abs/1802.08232“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>earlier work</a> on measuring unintended memorization in neural networks, which intentionally inserts unique, random <em class=„markup–em markup–p-em“>canary</em> sentences into the training data and assesses the canaries’ impact on the trained model. In this case, the insertion of a single random canary sentence is sufficient for that canary to be completely memorized by the non-private model. However, the model trained with differential privacy is indistinguishable in the face of any single inserted canary; only when the same random sequence is present many, many times in the training data, will the private model learn anything about it. Notably, this is true for all types of machine-learning models (e.g., see the figure with rare examples from MNIST training data above) and remains true even when the mathematical, formal upper bound on the model’s privacy is far too large to offer any guarantees in theory.</p><figure name=„5410“ id=„5410“ class=„graf graf–figure graf-after–p“><div class=„aspectRatioPlaceholder is-locked“ style=„max-width: 700px; max-height: 290px;“><div class=„aspectRatioPlaceholder-fill“ style=„padding-bottom: 41.4%;“/><img class=„graf-image“ data-image-id=„1*rsLWw9AtVESxXWhl33DQEg.png“ data-width=„1254“ data-height=„519“ data-is-featured=„true“ data-action=„zoom“ data-action-value=„1*rsLWw9AtVESxXWhl33DQEg.png“ src=„https://cdn-images-1.medium.com/max/1600/1*rsLWw9AtVESxXWhl33DQEg.png“/></div></figure><p name=„a08d“ id=„a08d“ class=„graf graf–p graf-after–figure“>TensorFlow Privacy can prevent such memorization of rare details and, as visualized in the figure above, can guarantee that two machine-learning models will indistinguishable whether or not some examples (e.g., some user’s data) was used in their training.</p><h3 name=„58ef“ id=„58ef“ class=„graf graf–h3 graf-after–p“>Next steps and further reading</h3><p name=„fc63“ id=„fc63“ class=„graf graf–p graf-after–h3“>To get started with TensorFlow Privacy, you can check out the examples and tutorials in the <a href=„https://github.com/tensorflow/privacy/tree/master/tutorials“ data-href=„https://github.com/tensorflow/privacy/tree/master/tutorials“ class=„markup–anchor markup–p-anchor“ rel=„nofollow noopener“ target=„_blank“>GitHub repository</a>. In particular, these include a detailed tutorial for how to perform differentially-private training of the MNIST benchmark machine-learning task with traditional TensorFlow mechanisms, as well as the newer more <em class=„markup–em markup–p-em“>eager</em> approaches of TensorFlow 2.0 and Keras.</p><p name=„8c07“ id=„8c07“ class=„graf graf–p graf-after–p“>The crucial, new steps required to utilize TensorFlow Privacy is to set three new hyperparameters that control the way gradients are created, clipped, and noised. During training, differential privacy is ensured by optimizing models using a modified stochastic gradient descent that averages together multiple gradient updates induced by training-data examples, clips each gradient update to a certain maximum norm, and adds a Gaussian random noise to the final average. This style of learning places a maximum bound on the effect of each training-data example, and ensures that no single such example has any influence, by itself, due to the added noise. Setting these three hyperparameters can be an art, but the TensorFlow Privacy repository includes guidelines for how they can be selected for the concrete examples.</p><p name=„895c“ id=„895c“ class=„graf graf–p graf-after–p“>We intend for TensorFlow Privacy to develop into a hub of best-of-breed techniques for training machine-learning models with strong privacy guarantees. Therefore, we encourage all interested parties to get involved, e.g., by doing the following:</p><ul class=„postList“><li name=„7b45“ id=„7b45“ class=„graf graf–li graf-after–p“>Read further about differential privacy and its application to machine learning in <a href=„https://desfontain.es/privacy/almost-differential-privacy.html“ data-href=„https://desfontain.es/privacy/almost-differential-privacy.html“ class=„markup–anchor markup–li-anchor“ rel=„nofollow noopener“ target=„_blank“>this</a> or <a href=„http://www.cleverhans.io/privacy/2018/04/29/privacy-and-machine-learning.html“ data-href=„http://www.cleverhans.io/privacy/2018/04/29/privacy-and-machine-learning.html“ class=„markup–anchor markup–li-anchor“ rel=„nofollow noopener“ target=„_blank“>that</a> blog post.</li><li name=„47cd“ id=„47cd“ class=„graf graf–li graf-after–li“>For practitioners, try applying TensorFlow Privacy on your own machine-learning models, and experiment with the balance between privacy and utility by tuning hyperparameters, model capacity and architectures, activation functions, etc.</li><li name=„b4ac“ id=„b4ac“ class=„graf graf–li graf-after–li“>For researchers, try advancing the state of the art in real-world machine learning with strong privacy guarantees by improved analysis, e.g. of <a href=„https://arxiv.org/abs/1811.07971“ data-href=„https://arxiv.org/abs/1811.07971“ class=„markup–anchor markup–li-anchor“ rel=„nofollow noopener“ target=„_blank“>model parameter selection</a>.</li><li name=„5ece“ id=„5ece“ class=„graf graf–li graf-after–li“>Contribute to TensorFlow Privacy by submitting pull requests.</li><li name=„46e9“ id=„46e9“ class=„graf graf–li graf-after–li“>Ask questions and share your comments or concerns by filing issues on <a href=„https://github.com/tensorflow/privacy“ data-href=„https://github.com/tensorflow/privacy“ class=„markup–anchor markup–li-anchor“ rel=„nofollow noopener“ target=„_blank“>GitHub</a>.</li></ul><h3 name=„9afb“ id=„9afb“ class=„graf graf–h3 graf-after–li“>Acknowledgements</h3><p name=„8ef8“ id=„8ef8“ class=„graf graf–p graf-after–h3 graf–trailing“>We’d like to thank Galen Andrew, Nicholas Carlini, Steve Chien, Brendan McMahan, Ilya Mironov, and Nicolas Papernot for their contributions to TensorFlow Privacy.</p></div></div> </html>
Falls nicht anders bezeichnet, ist der Inhalt dieses Wikis unter der folgenden Lizenz veröffentlicht: CC Attribution-Noncommercial-Share Alike 4.0 International