Pycon Israel 2021

Model-Agnostic Interpretation - Beyond Shap and Lime
2021-05-03, 10:30–10:55, PyData Track 1

I’ll discuss an interpretation framework that allows use of the features’ distribution to understand the direction of the feature’s impact. The concept is derived from ideas formulated in Pearl’s analysis of causality in his book “the book of why”.


The subject of interpretability becomes very important as models grow more and more complex but humans need to reason them. Since we don’t want to be blocked by the model’s algorithm (for example, if we want to bag several models), the community offers solutions that are based on alternatives analysis - local assessment, shuffling features, etc.

In this talk, I’ll offer a framework that allows use of the features’ distribution to understand the direction of the feature’s impact, both on the entire sample’s level and for specific observations. The inner workings of this method is highly intuitive and straightforward, and its concept is derived from ideas formulated in Judea Pearl’s analysis of causality (check out “the book of why” for more info).

I’ll present a specific use case of tabular data from Bluevine, and compare its performance to available solutions. I’ll also mention directions for applying a similar method to additional fields.

Nathalie Hauser,
Data Science Manager @Bluevine


Session language – English Target audience – Data Scientists