# ROC-AUC

Note

"You are writing a book because you are not entirely satisfied with the available texts." -- George Casella

## When&Where&Who&Why¶

Tip

The ROC curve was first used during World War II for the analysis of radar signals before it was employed in signal detection theory. [44] Following the attack on Pearl Harbor in 1941, the United States army began new research to increase the prediction of correctly detected Japanese aircraft from their radar signals. For these purposes they measured the ability of a radar receiver operator to make these important distinctions, which was called the Receiver Operating Characteristic. --Wikipedia

Note

ROC曲线最早是运用在军事上的,后来逐渐运用到医学领域,并于20世纪80年代后期被引入机器学习领域。相传在第二次 世界大战期间,雷达兵的任务之一就是死死地盯住雷达显示器,观察是否有敌机来袭。理论上讲,只要有敌机来袭,雷达屏幕上 就会出现相应的信号。但是实际上,如果飞鸟出现在雷达扫描区域时,雷达屏幕上有时也会出现信号。这种情况令雷达兵烦恼不 已,如果过于谨慎,凡是有信号就确定为敌机来袭,显然会增加误报风险;如果过于大胆,凡是信号都认为是飞鸟,又会增加漏报的风险。每个雷达兵都竭尽所能地研究飞鸟信号和飞机信号之间的区别,以便增加预报的准确性。但问题在于,每个雷达兵都 有自己的判别标准,有的雷达兵比较谨慎,容易出现误报;有的雷达兵则比较胆大,容易出现漏报。

## What¶

Note

A receiver operating characteristic curve, or ROC curve, is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. --Wikipedia

Wiki 上明确地给出我前面说的“某样东西”是一个"binary classifier sysytem"，即一个二分类系统 (再次强调，请不要局限在 ML 里面的二分类...一个分辨灯泡好坏的仪器同样是一个二分类系统，这在质量管理中的例子很多).

Note

TP, FP, TN, FN: True Positive, False Positive, True Negative, False Negative

TPR: True Postive Rate, Recall, Sensitivity, $$\(TPR = \frac{TP}{TP+FN}$$\)

FPR: False Positive Rate, Fall-out, $$FPR=\frac{FP}{FP+TN}$$

ROC 定义横轴为 FPR，纵轴为 TPR。进而问题转化成如何从Threshold $$T$$得到对应点的横纵座标，即$$(FPR_T，TPR_T)$$.从某个集合$$S_T$$中取不同的阈值$$T$$，就得到一系列的横纵座标，也就是 ROC 曲线上的一系列点$$\{(FPR_T, TPR_T) | T \in S_T\}$$, 也就得到了 ROC 曲线。

Tip

## AUC¶

AUC 的概率解释在 Wikipedia 中有详细的推导。

\begin{aligned} &T P R(T): T \rightarrow y(x)\\ &F P R(T): T \rightarrow x\\ &\mathrm{TPR}(T)=\int_{T}^{\infty} f_{1}(x) d x\\ &\operatorname{FPR}(T)=\int_{T}^{\infty} f_{0}(x) d x\\ &AUC=\int_{x=0}^{1} \operatorname{TPR}\left(\operatorname{FPR}^{-1}(x)\right) d x=\int_{\infty}^{-\infty} \operatorname{TPR}(T) \operatorname{FPR}^{\prime}(T) d T=\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} I\left(T^{\prime}>T\right) f_{1}\left(T^{\prime}\right) f_{0}(T) d T^{\prime} d T=P\left(X_{1}>X_{0}\right) \end{aligned}

When using normalized units, the area under the curve (often referred to as simply the AUC) is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one (assuming 'positive' ranks higher than 'negative')