模型可解释性

FingerLiu

对于模型可解释性的资料较少,这里统一做个整理。
如果想系统性学习,推荐阅读Interpretable machine learning 这本书。

What

可解释性是人可以理解系统决策的一个度量(Interpretability is the degree to which a human can understand the cause of a decision.)

Why

随着大量科学家在机器学习领域的持续投入,如今各种模型五花八门,且毋庸置疑这些模型在生产环境都产生的一定的效果。

但计算机系统做出的这些预测或决策的原因并无法被人类感知,因而人类无法直观的感知这些预测的效果或原因。

模型都是对现实世界的模拟,但是都不是 100% 的还原,因此没有一个单一 metric(如 auc) 能描述模型在生成环境中的真实表现。

此外机器学习系统并不知道自己在预测什么内容。

这个问题在某些领域可能非常重要(例如临床医学,反欺诈)。

How(理论)

  • 使用有自解释性的模型(WB)
  • 基于模型不可知论的黑盒探测(BB)
  • 深度学习/神经网络模型的可解释性

自解释性的模型

  • 线性回归模型
  • 逻辑回归
  • 决策树

白盒解释,需要知道模型具体实现,以及训练数据,甚至可能需要改模型代码。

黑盒探测

pdp

  • Partial Dependence Plot (PDP)
  • Global Surrogate
  • Local Surrogate Model-Agnostic Method(LIME)
  • Anchors(If-Then-That)
  • Shapley Values
  • SHAP

PDP

A partial dependence plot can show whether the relationship between the target and a feature is linear, monotonic or more complex.

Advantages: intuitive
Disadvantages:The assumption of independence is the biggest issue with PD plots.

pdp

Global Surrogate

A global surrogate model is an interpretable model that is trained to approximate the predictions of a black box model. 

Advantages: flexible.Any interpretable models can be used. 
Disadvantages:You have to be aware that you draw conclusions about the model and not about the data, since the surrogate model never sees the real outcome.

Local Surrogate Model-Agnostic Method(LIME)

Local surrogate models are interpretable models that are used to explain individual predictions of black box machine learning models.

Advantages: They make human-friendly explanations.LIME is one of the few methods that works for tabular data, text and images.
Disadvantages: The correct definition of the neighborhood differs./instability.

lime

Anchor

A rule anchors a prediction if changes in other feature values do not affect the prediction. 
Anchors utilizes reinforcement learning techniques in combination with a graph search algorithm to reduce the number of model calls (and hence the required runtime) to a minimum while still being able to recover from local optima. 

Advantages: Like Lime. 
    Anchors are subsettable(shown as example).
    Works for non-linear or complex in an instance’s neighborhood(reinforcement learning).
    Can be parallelized.
    
Disadvantages: highly configurable.
    many scenarios require discretization.
    many calls to the ML model.

anchor

Shapley Values

SHAP

explore timeline

PDP(2001) --> LIME(2016) --> Anchors(2018)
Shapley Values(2014) --> SHAP(2016)

深度学习/神经网络模型

深度学习/神经网络模型的解释可以使用黑盒探测来实现,除此之外还有一些专门针对神经网络的可解释性科研探索。

  • Feature Visualization
  • Network dissection
  • Pixel Attribution (Saliency Maps)

When to use

  • Your model makes significant impact.
  • When the problem is not well studied, or explore in a very new area.

In Action(实战)

python: sklearn、keras、alibi

R: iml

seldon 是一个模型生命周期管理的系统,有点类似于我们的 PAS 加一部分 DAG 的功能,他们基于上述 LIME 和 Anchor 等实现了一套模型解释和异常检测的框架并开源了,我们可以借鉴、探索下。

模型 outlier, adversarial and drift

https://github.com/SeldonIO/a...

模型 inspection and interpretation

https://github.com/SeldonIO/a...

具体参考 explain.ipynb

vs

阅读 745

FingerLiu
先广后精。Explore the whole world,and then do one thing but do it best.

先广后精。Explore the whole world,and then do one thing but do it best.

562 声望
28 粉丝
0 条评论

先广后精。Explore the whole world,and then do one thing but do it best.

562 声望
28 粉丝
文章目录
宣传栏