Interpretable machine learning: definitions, methods, and applications

Abstract

📜 Abstract

Machine-learning models have demonstrated great success in learning complex patterns that enable them to make predictions about unobserved data. In addition to using models for prediction, the ability to interpret what a model has learned is receiving an increasing amount of attention. However, this increased focus has led to considerable confusion about the notion of interpretability. In particular, it is unclear how the wide array of proposed interpretation methods are related, and what common concepts can be used to evaluate them. We aim to address these concerns by defining interpretability in the context of machine learning and introducing the Predictive, Descriptive, Relevant (PDR) framework for discussing interpretations. The PDR framework provides three overarching desiderata for evaluation: predictive accuracy, descriptive accuracy and relevancy, with relevancy judged relative to a human audience. Moreover, to help manage the deluge of interpretation methods, we introduce a categorization of existing techniques into model-based and post-hoc categories, with sub-groups including sparsity, modularity and simulatability. To demonstrate how practitioners can use the PDR framework to evaluate and understand interpretations, we provide numerous real-world examples. These examples highlight the often under-appreciated role played by human audiences in discussions of interpretability. Finally, based on our framework, we discuss limitations of existing methods and directions for future work. We hope that this work will provide a common vocabulary that will make it easier for both practitioners and researchers to discuss and choose from the full range of interpretation methods.

Description

✨ Summary

The paper defines interpretable machine learning as using machine-learning models to extract relevant knowledge about domain relationships in data. It emphasizes that interpretability is audience- and problem-dependent rather than a single intrinsic property of a model.

Its central contribution is the Predictive, Descriptive, Relevant (PDR) framework:

Predictive accuracy: whether the model adequately represents the underlying data relationships.
Descriptive accuracy: whether the interpretation faithfully captures what the model has learned.
Relevancy: whether the interpretation provides useful insight for a specific audience and domain problem.

The paper places interpretation within a data-science life cycle and distinguishes between model-based interpretability, which constrains the model to make its learned relationships easier to understand, and post-hoc interpretability, which analyzes an already-trained model without changing it. Model-based methods include sparsity, simulatability, modularity, domain-based feature engineering, and model-based feature engineering. Post-hoc methods include dataset-level and prediction-level analyses, feature and interaction importance, statistical summaries, visualizations, influence functions, and alternative explanation formats.

A recurring methodological requirement is stability: interpretations should remain meaningful under appropriate perturbations of the data, model, or training procedure. The authors also distinguish interpretability from causal inference, noting that ordinary interpretation generally describes associations rather than causal effects.

The paper identifies unresolved problems in evaluating descriptive accuracy and relevancy. It recommends simulation studies, comparison with prior scientific knowledge, direct use in domain problems, and human-subject studies as partial evaluation strategies. It further argues that future work should develop more accurate intrinsically interpretable models, better feature-engineering tools, improved formats for explaining black-box models, and methods that use interpretations to correct or improve predictive models.

Subsequent research has cited the paper as a framework for organizing interpretability terminology and methods. Later taxonomy and survey work uses it when discussing definitions, evaluation criteria, model-based versus post-hoc approaches, and the role of human audiences in explanation design. (springerlink.fh-diploma.de) A 2026 bibliometric analysis also includes the paper among influential works in the explainable-AI literature, indicating continued use as a reference point for mapping the field. (onlinelibrary.wiley.com)