Distribution shifts—where the training distribution differs from the test distribution—can
substantially degrade the accuracy of machine learning (ML) systems deployed in the wild. …

Standard training via empirical risk minimization (ERM) can produce models that achieve low
error on average but high error on minority groups, especially in the presence of spurious …

Overparameterized neural networks can be highly accurate on average on an iid test set yet
consistently fail on atypical groups of the data (eg, by learning spurious correlations that …

How can we explain the predictions of a black-box model? In this paper, we use influence
functions—a classic technique from robust statistics—to trace a model’s prediction through the …

AI is undergoing a paradigm shift with the rise of models (eg, BERT, DALL-E, GPT-3) that are
trained on broad data at scale and are adaptable to a wide range of downstream tasks. We …

We seek to learn models that we can interact with using high-level concepts: if the model
did not think there was a bone spur in the x-ray, would it still predict severe arthritis? State-of-the…

The coronavirus disease 2019 (COVID-19) pandemic markedly changed human mobility
patterns, necessitating epidemiological models that can capture the effects of these changes in …

Abstract Machine learning models trained on data from the outside world can be corrupted by
data poisoning attacks that inject malicious points into the models’ training sets. A common …

For machine learning systems to be reliable, we must understand their performance in
unseen, out-of-distribution environments. In this paper, we empirically show that out-of-distribution …

Peer and self-assessment offer an opportunity to scale both assessment and learning to global
classrooms. This article reports our experiences with two iterations of the first large online …