In the recent years, machine learning technologies are increasing in popularity among variety of different fields. The vast amounts of data allow data scientists to play around and try to answer questions that are impossible to accurately calculate without such technologies. Some of these questions can be highly important and sensitive – for example, deciding whether a loan application will default, or whether an applicant should be considered for the job. There are variety of models that can predict such outcomes, however, is their prediction generalizable and free of biases?

A classical example of this is a neural network classification model, that was built to identify whether in a given picture there is a husky or a wolf. Surprisingly, the model was performing with a high accuracy and predicting the correct animals. However, researchers quickly realized, that something was wrong with it. The model, was basing its decisions purely on one aspect – whether the picture contains snow. Well, that is quite a logical explanation, as pictures of wolves do indeed have usually snow in them, however, given the explanation for this reasoning, I doubt that anyone would want to apply this biased model in a real life situation.

In making predictions, there is often a trade-off between interpretability and model accuracy. When a linear model predicts an outcome, the prediction is just input variables multiplied by different weights – these can be easily explained. However, when using a more advanced model, such as gradient boosting classifier or neural networks, this interpretation becomes complicated.
In recent years, however, there has been some effort to tackle this; one of them is the so called Local Interpretable Model-agnostic Explanations (LIME). Without going too deep in the theoretical details (I will provide some links for further information) LIME is essentially a technique, that helps better explain the reasoning behing any model‘s particular prediction, locally – meaning that it only allows to analyze a small portion of the network/model.

In practice, LIME allows to understand predicitons, by visualizing the features of a given observation that had the most impact in making that particular prediction. Returning back to our wolf/husky identifier – in a given picture, LIME will highlight the snow that that there either is, or is not, hence explaining the behaviour of that particular model.

The importance of this, is that it allows data scientists to better evaluate their models and their applicability. When a model will predict a loan applicant as ‚likely to default‘, and a bank makes the decision to deny him that loan – managers will be able to explain to the person what are the exact reasons. In a critical situations, we cannot simply use black boxes that give us answers to our simplified questions – we also need to know the reasoning.

Sources/Further information:
Video: Interpretable Machine Learning Using LIME Framework – Kasia Kulma (PhD), Data Scientist, Aviva
Article: Guide to Interpretable Machine Learning