The Receiver Operating Characteristic Curve

Now, consider how P(Correct Acceptance|a) and P(False Alarm|a) change as the forager changes the decision threshold a. Suppose that our insectivorous bird picks a threshold a value—say, a—that leads it to always accept beetles regardless of their color. In this case, our forager will never miss a truly tasty beetle [P(Correct Acceptance|a) = 1], but the price of this advantage is that it always incorrectly accepts noxious beetles [P(False Alarm|s) = 1]. At the other end of the spectrum, imagine that our insectivorous bird picks an a value—say, a—that causes it to reject everything. Then the forager will never accept a noxious beetle [P(False Alarm|a) = 0], but it will always reject tasty beetles [P(Correct Acceptance|a) = 0]. As the parameter a changes from values specifying "always accept" to values specifying "always reject," it determines noxious beetles tasty beetles noxious beetles tasty beetles

Figure 2.1. The relationship between P(False Alarm) and P(Correct Acceptance). P(False Alarm) is the area underthe lower (noxious beetle) curve that is also above a (light shading). P(Correct Acceptance is the area underthe higher (tasty beetle) curve that is above a (darker shading).

Figure 2.1. The relationship between P(False Alarm) and P(Correct Acceptance). P(False Alarm) is the area underthe lower (noxious beetle) curve that is also above a (light shading). P(Correct Acceptance is the area underthe higher (tasty beetle) curve that is above a (darker shading).

a relationship between P(False Alarmja) and P(Correct Acceptanceja). This relationship, called the receiver operating characteristic (ROC) curve, is a fundamental part of our analysis because it gives a powerful and concise summary of the constraint imposed by imperfect discrimination. The receiver operating characteristic curve focuses our attention on the trade-offbetween high acceptance rates that lead to few misses but frequent false alarms, and high rejection rates that lead to few false alarms but frequent misses.

We can take the logic above a bit further to show how the entire receiver operating characteristic curve can be constructed. Figure 2.1 shows two overlapping color (green-to-black) distributions. The distribution on the right shows the (blacker) colors of tasty beetles, and the distribution on the left shows the (greener) colors of noxious beetles. If we choose an acceptance threshold a, the probabilities of acceptance are the areas under the curves above a, as indicated in the figure. P(Correct Acceptanceja) is the area above a under the upper "tasty beetle" curve, and P(False Alarmja) is the analogous area above a under the lower "noxious beetle" curve. As a increases, the two probabilities of acceptance move in concert, tracing out a receiver operating characteristic curve, as figure 2.2 shows.

A comparison of figures 2.2A and 2.2B shows how receiver operating characteristic curves differ between easy and difficult discrimination problems. Part A shows a case in which the two distributions are well separated, making this an easy discrimination problem, because we can easily choose an a value that rejects most noxious beetles and accepts most tasty beetles. The figure shows how this situation leads to a strongly "bowed out" receiver

Was this article helpful?

0 0

Post a comment