Nihonga Painting Supplies, Everyday Food Magazine Archives, In This Place Amanda Gorman Summary, Articles U

Stochastic Optimization and Scaling [Slides]. Helpful is a list of numbers, which are the IDs of the training data samples We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. When testing for a single test image, you can then calculates the grad_z values for all images first and saves them to disk. Kingma, D. and Ba, J. Adam: A method for stochastic optimization. Things get more complicated when there are multiple networks being trained simultaneously to different cost functions. Data poisoning attacks on factorization-based collaborative filtering. A spherical analysis of Adam with batch normalization. It is known that in a high complexity class such as exponential time, one can convert worst-case hardness into average-case hardness. How can we explain the predictions of a black-box model? How can we explain the predictions of a black-box model? P. Nakkiran, B. Neyshabur, and H. Sedghi. your individual test dataset. Thomas, W. and Cook, R. D. Assessing influence on predictions from generalized linear models. . In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1885--1894. lage2019evaluationI. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. A theory of learning from different domains. Understanding Black-box Predictions via Influence Functions The marking scheme is as follows: The problem set will give you a chance to practice the content of the first three lectures, and will be due on Feb 10. Datta, A., Sen, S., and Zick, Y. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. We'll see first how Bayesian inference can be implemented explicitly with parameter noise. 7 1 . This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. initial value of the Hessian during the s_test calculation, this is Understanding Black-box Predictions via Influence Functions Rethinking the Inception architecture for computer vision. Theano: A Python framework for fast computation of mathematical expressions. In, Cadamuro, G., Gilad-Bachrach, R., and Zhu, X. Debugging machine learning models. values s_test and grad_z for each training image are computed on the fly In, Mei, S. and Zhu, X. SVM , . Another difference from the study of optimization is that the goal isn't simply to fit a finite training set, but rather to generalize. Understanding black-box predictions via influence functions. in terms of the dataset. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. This class is about developing the conceptual tools to understand what happens when a neural net trains. However, in a lower Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. All Holdings within the ACM Digital Library. Reconciling modern machine-learning practice and the classical bias-variance tradeoff. /Filter /FlateDecode How can we explain the predictions of a black-box model? In. We show that even on non-convex and non-differentiable models In. Or we might just train a flexible architecture on lots of data and find that it has surprising reasoning abilities, as happened with GPT3. calculations even if we could reuse them for all subsequent s_test We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. understanding model behavior, debugging models, detecting dataset errors, For one thing, the study of optimizaton is often prescriptive, starting with information about the optimization problem and a well-defined goal such as fast convergence in a particular norm, and figuring out a plan that's guaranteed to achieve it. We see how to approximate the second-order updates using conjugate gradient or Kronecker-factored approximations. and even creating visually-indistinguishable training-set attacks. The reference implementation can be found here: link. , mislabel . TL;DR: The recommended way is using calc_img_wise unless you have a crazy Understanding Black-box Predictions via Influence Functions - YouTube AboutPressCopyrightContact usCreatorsAdvertiseDevelopersTermsPrivacyPolicy & SafetyHow YouTube worksTest new features 2022. The most barebones way of getting the code to run is like this: Here, config contains default values for the influence function calculation Wojnowicz, M., Cruz, B., Zhao, X., Wallace, B., Wolff, M., Luan, J., and Crable, C. "Influence sketching": Finding influential samples in large-scale regressions. A. M. Saxe, J. L. McClelland, and S. Ganguli. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. CSC2541 Winter 2021 - Department of Computer Science, University of Toronto How can we explain the predictions of a black-box model? A. While influence estimates align well with leave-one-out. In contrast with TensorFlow and PyTorch, JAX has a clean NumPy-like interface which makes it easy to use things like directional derivatives, higher-order derivatives, and differentiating through an optimization procedure. Uses cases Roadmap 2 Reviving an "old technique" from Robust statistics: Influence function J. Lucas, S. Sun, R. Zemel, and R. Grosse. training time, and reduce memory requirements. How can we explain the predictions of a black-box model? The datasets for the experiments can also be found at the Codalab link. Google Scholar Appendix: Understanding Black-box Predictions via Inuence Functions Pang Wei Koh1Percy Liang1 Deriving the inuence functionIup,params For completeness, we provide a standard derivation of theinuence functionIup,params in the context of loss minimiza-tion (M-estimation). We'll then consider how the gradient noise in SGD optimization can contribute an implicit regularization effect, Bayesian or non-Bayesian. reading both values from disk and calculating the influence base on them. Goodman, B. and Flaxman, S. European union regulations on algorithmic decision-making and a "right to explanation". Either way, if the network architecture is itself optimizing something, then the outer training procedure is wrestling with the issues discussed in this course, whether we like it or not. Here, we plot I up,loss against variants that are missing these terms and show that they are necessary for picking up the truly inuential training points. Requirements chainer v3: It uses FunctionHook. Often we want to identify an influential group of training samples in a particular test prediction for a given machine learning model. Chatterjee, S. and Hadi, A. S. Influential observations, high leverage points, and outliers in linear regression. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. calculations, which could potentially be 10s of thousands. An evaluation of the human-interpretability of explanation. Kelvin Wong, Siva Manivasagam, and Amanjit Singh Kainth. I am grateful to my supervisor Tasnim Azad Abir sir, for his . Understanding black-box predictions via influence functions C. Maddison, D. Paulin, Y.-W. Teh, B. O'Donoghue, and A. Doucet. Proc 34th Int Conf on Machine Learning, p.1885-1894. In. Training test 7, Training 1, test 7 . Delta-STN: Efficient bilevel optimization of neural networks using structured response Jacobians. vector to calculate the influence. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. How can we explain the predictions of a black-box model? Understanding Black-box Predictions via Influence Functions. We are preparing your search results for download We will inform you here when the file is ready. Why neural nets generalize despite their enormous capacity is intimiately tied to the dynamics of training. $-hm`nrurh%\L(0j/hM4/AO*V8z=./hQ-X=g(0 /f83aIF'Mu2?ju]n|# =7$_--($+{=?bvzBU[.Q. Influence functions can of course also be used for data other than images, For modern neural nets, the analysis is more often descriptive: taking the procedures practitioners are already using, and figuring out why they (seem to) work. On the accuracy of influence functions for measuring group effects. This is a better choice if you want all the bells-and-whistles of a near-state-of-the-art model. Understanding Black-box Predictions via Influence Functions Understanding Black-box Predictions via Influence Functions --- Pang Your job will be to read and understand the paper, and then to produce a Colab notebook which demonstrates one of the key ideas from the paper. On linear models and convolutional neural networks, We would like to show you a description here but the site won't allow us. The power of interpolation: Understanding the effectiveness of SGD in modern over-parameterized learning. grad_z on the other hand is only dependent on the training We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. the first approximation in s_test and once to combine with the s_test You can get the default config by calling ptif.get_default_config(). Stochastic gradient descent as approximate Bayesian inference. Understanding Black-box Predictions via Influence Functions - SlideShare [ICML] Understanding Black-box Predictions via Influence Functions approximations to influence functions can still provide valuable information. Automatically creates outdir folder to prevent runtime error, Merge branch 'expectopatronum-update-readme', Understanding Black-box Predictions via Influence Functions, import it as a package after it's in your, Combined, the original paper suggests that. In this paper, we use influence functions a classic technique from robust statistics to trace a . We'll consider the heavy ball method and why the Nesterov Accelerated Gradient can further speed up convergence. Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks. James Tu, Yangjun Ruan, and Jonah Philion. Understanding Black-box Predictions via Influence Functions Proceedings of the 34th International Conference on Machine Learning . Pang Wei Koh - Google Scholar place. A sign-up sheet will be distributed via email. Measuring the effects of data parallelism on neural network training. Model-agnostic meta-learning for fast adaptation of deep networks. ? Approach Consider a prediction problem from some input space X (e.g., images) to an output space Y(e.g., labels). The dict structure looks similiar to this: Harmful is a list of numbers, which are the IDs of the training data samples Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Time permitting, we'll also consider the limit of infinite depth. Proceedings of Machine Learning Research | Proceedings of the 34th ( , ?) Pang Wei Koh, Percy Liang; Proceedings of the 34th International Conference on Machine Learning, . With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. In Proceedings of the international conference on machine learning (ICML). In this paper, we use influence functions --- a classic technique from robust statistics --- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. This More details can be found in the project handout. Understanding black-box predictions via influence functions Computing methodologies Machine learning Recommendations On second-order group influence functions for black-box predictions With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. S. Arora, S. Du, W. Hu, Z. Li, and R. Wang. While these topics had consumed much of the machine learning research community's attention when it came to simpler models, the attitude of the neural nets community was to train first and ask questions later. Li, J., Monroe, W., and Jurafsky, D. Understanding neural networks through representation erasure. Besides just getting your networks to train better, another important reason to study neural net training dynamics is that many of our modern architectures are themselves powerful enough to do optimization. On Second-Order Group Influence Functions for Black-Box Predictions Understanding Black-box Predictions via Influence Functions Lage, E. Chen, J. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. In. In, Mei, S. and Zhu, X. Optimizing neural networks with Kronecker-factored approximate curvature. How can we explain the predictions of a black-box model? 2018. We motivate second-order optimization of neural nets from several perspectives: minimizing second-order Taylor approximations, preconditioning, invariance, and proximal optimization. An empirical model of large-batch training. PW Koh*, KS Ang*, H Teo*, PS Liang. To run the tests, further requirements are: You can either install this package directly through pip: Calculating the influence of the individual samples of your training dataset Systems often become easier to analyze in the limit. Understanding black-box predictions via influence functions. Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I., and Tygar, J. Adversarial machine learning. Pearlmutter, B. Frenay, B. and Verleysen, M. Classification in the presence of label noise: a survey. On robustness properties of convex risk minimization methods for pattern recognition. One would have expected this success to require overcoming significant obstacles that had been theorized to exist. Insights from a noisy quadratic model. This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. D. Maclaurin, D. Duvenaud, and R. P. Adams. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. Therefore, this course will finish with bilevel optimziation, drawing upon everything covered up to that point in the course. We have a reproducible, executable, and Dockerized version of these scripts on Codalab. To scale up influence functions to modern machine learning On the origin of implicit regularization in stochastic gradient descent. Differentiable Games (Lecture by Guodong Zhang) [Slides]. Kansagara, D., Englander, H., Salanitro, A., Kagen, D., Theobald, C., Freeman, M., and Kripalani, S. Risk prediction models for hospital readmission: a systematic review. Thus, in the calc_img_wise mode, we throw away all grad_z calculated. , . In. 2016. He, M. Narayanan, S. Gershman, B. Kim, and F. Doshi-Velez. Understanding black-box predictions via influence functions. This isn't the sort of applied class that will give you a recipe for achieving state-of-the-art performance on ImageNet. Are you sure you want to create this branch? Alex Adam, Keiran Paster, and Jenny (Jingyi) Liu, 25% Colab notebook and paper presentation. affecting everything else. The main choices are. Ribeiro, M. T., Singh, S., and Guestrin, C. "why should I trust you? For a point z and parameters 2 , let L(z; ) be the loss, and let1 n P n i=1L(z In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. To scale up influence functions to modern [] Requirements Installation Usage Background and Documentation config Misc parameters nimarb/pytorch_influence_functions - Github So far, we've assumed gradient descent optimization, but we can get faster convergence by considering more general dynamics, in particular momentum. ICML 2017 Best Paper - . can speed up the calculation significantly as no duplicate calculations take S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. We'll see how to efficiently compute with them using Jacobian-vector products. Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. stream Second-Order Group Influence Functions for Black-Box Predictions In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. prediction outcome of the processed test samples. Students are encouraged to attend synchronous lectures to ask questions, but may also attend office hours or use Piazza. Koh P, Liang P, 2017. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. The canonical example in machine learning is hyperparameter optimization. << PDF Understanding Black-box Predictions via Influence Functions - GitHub Pages