Date/Time Date(s) - 10/10/20253:30 pm - 4:30 pm
Dr. Giles Hooker Professor, Department of Statistics & Data Science University of Pennsylvania Research Area: statistical methods using dynamical systems models, functional data analysis, and statistical aspects of fair and interpretable machine learning
Location: HH-305
Title: There is No Free Variable Importance: Bias and Uncertainty Quantification for Explainable AI
Abstract: The field of machine learning — loosely defined as nonparametric statistical modeling — has become enormously successful over the past fifty years, partly by forgoing the parametric models familiar to statisticians. A consequence of this philosophy has been that these methods result in algebraically-complex models that provide little human-accessible insight into the workings of the model, or what it might say about the underlying processes generating the data. As these methods have been taken up in high-stakes decision making, demands to “x-ray the black box” have become more prevalent, resulting in a wide variety of approaches to understand what signal the model is capturing, or to provide explanations of individual predictions.
Unfortunately, many of these methods produce results that can lead to mistaken conclusions about the model, or the underlying processes, or both. This talk reviews two sources of error: distorting the covariate distribution beyond the range where the model performs well, and estimating structured surrogates using insufficient data. We show that many popular interpretation/explanation methods suffer from these, potentially resulting in mistaken conclusions or advice, and review the properties necessary to generate reliable explanation or interpretation.