OOD Detection

Original: https://zhuanlan.zhihu.com/p/102870562

What is OOD Detection?

OOD(out-of-distribution) detection aims to detect an OOD sample, which is the opposite of ID(in-distribution) sample.

In traditional machine learning methods, the training and test data are assumed to be independent identical distribution(IID). However, in real-world scenario, test samples are likely to be OOD, or outlier. Traditional deep learning models often consider an OOD sample as a class of ID samples with high confidence, which is not reasonable. Thus it is meaningful for models to recognize OOD samples, especially for AI security areas.

Methods

Existing OOD Detection methods can be roughly divided into 4 categories:

• Softmax-based methods.
• Uncertainty methods.
• Generative model methods.
• Classifier methods.

  Softmax-based Methods

  Softmax-based methods get the softmax values of the outputs of a pre-trained model. And find out how OOD and ID samples distribute by statistical analysis. It aims to enlarge the difference between OOD and ID distributions.

Softmax-based methods are simple, entirely training-free yet effective, without the need to change model structure.

SMOOD

Paper URL: https://arxiv.org/abs/1610.02136

SMOOD(SoftMax OOD) is the very first work of OOD Detection task. It proposes an OOD baseline method. Its main insights are:

1. Correctly classified examples tend to have greater maximum softmax probabilities than erroneously classified and out-of-distribution examples, allowing for their detection.
2. Proposed a simple yet effective baseline method to detect whether a sample is mis-classified or out-of-distribution.

The above tables show that using simply max softmax probabilities to in- or out-of-distribution samples is effective. But it can not tell if the model wrongly classifies the sample.

In conclusion, SMOOD justifies the softmax output of the model differs when an in- or out-of-distribution sample is given. So they can be effectively discriminated by choosing a proper threshold.

ODIN

Paper URL: https://arxiv.org/abs/1706.02690

SMOOD is simple and effective. But the effect is not good enough. To achieve better results, ODIN proposes to enlarge the gap between in- and out-of-distribution softmax probabilities.

Based on above thoughts, it introduces two main methods:

• Temperature Scaling
• Input Preprocessing

Temperature Scaling

$$p_i(x;T)=\frac{\exp(f_i(x)/T)}{\sum_{j=1}^N\exp(f_i(x)/T)}$$

A $T$ that is large enough makes the softmax probabilities close enough to $\frac{1}{N}$.

Image Preprocessing

ODIN preprocesss the input by adding small perturbations:

$$\widetilde{x}=x-\epsilon\text{sign}(-\bigtriangledown_x\log(p_{\hat{y}}(x;T))$$

$x$ and $\epsilon$ denote the input sample and perturbation magnitude, respectively.
In adversarial examples(where $\widetilde{x}=x+\epsilon\text{sign}$), small perturbations are added to decrease the softmax score for the true label and force the model to make a wrong prediction. In ODIN, the goal is the opposite: it aims to increase the softmax score of any given input.
Why it works? ID sample confidence are increased dramatically while OOD sample confidence stays the same. Thus the confidence gap between ID and OOD samples is enlarged.

Uncertainty Methods

Uncertainty-based methods mainly learns the uncertainty of the model predictions. The uncertainty should be high given an OOD sample and low given an ID sample.

Learning Confidence for OOD Detection

Paper URL: https://arxiv.org/abs/1802.04865

Motivation. In this paper the authors propose an idea that a network can estimate its prediction confidence and ask for “hints”.

In order to estimate the confidence of the prediction, a confidence estimation branch is added in parallel with the original class prediction branch, as shown in the above figure. It receives the same input as the class prediction branch and outputs the confidence.
The confidence branch contains one or more fully-connected layers, with the final layer outputting a single scalar $c$ between 0 and 1 (parameterized as a sigmoid).
The output of the prediction and confidence branch would be

$$p, c = f(x, \Theta), p_i,c\in[0,1],\sum_{i=1}^Mp_i=1.$$

To “hint” the model, the softmax prediction probabilities are adjusted by interpolating between the original predictions $p$ and the target probability distribution $y$ during training, where the degree of interpolation is indicated by the network’s confidence $c$:

$$p_i' = c \cdot p_i + (1-c)y_i.$$

The loss function $\mathcal{L}_t$ should be as usual, except for that the prediction $p$ should be the modified prediction $p’$.

However, if the model always predict the confidence as $0$, the loss will always be the lowest. To prevent this, the confidence loss, a penalty is added to the loss function. This can be interpreted as a binary cross-entropy loss, where the target value is always 1 (i.e., we want the network to always be very confident):

$$\mathcal{L}_c=-\log(c).$$

So the final loss should be:

$$\mathcal{L} = \mathcal{L}_t + \lambda \cdot \mathcal{L}_c,$$

where $\lambda$ is a super parameter.

Multiple Semantic Label Representations

Paper URL: https://arxiv.org/abs/1808.06664