Finding the Optimal Discrimination Strategy

Now that we have the machinery of the receiver operating characteristic curve, we can find the "optimal" threshold a. We will simply quote the result here (Commons et al. 1991; Egan 1975; Gescheider 1985; Green and Swets 1966; Wiley 1994). We established above that the chosen value of the threshold a implicitly determines a point on the receiver operating characteristic curve. Of course, the reverse applies as well: for a given point on the receiver operating characteristic curve, we can find the corresponding a (doing this requires some very laborious algebra, but it is logically straightforward). So we will state our "solution" in terms of the receiver operating characteristic curve. The optimal point on the receiver operating characteristic is the point that has a slope equal to

m where p is the proportion of beetles that are tasty (so 1 — p are noxious), and the V terms come from the payoff table given above. This term, m*, will be a large number if noxious beetles are much more common than tasty beetles (p near zero), predicting that the solution should be on a steep part ofthe receiver operating characteristic curve (implying a high, generally "unaccepting," a value; fig. 2.3). If, instead, tasty beetles are more common (p near 1), then m* will be small, and the solution will be on the shallower (upper) portion of the receiver operating characteristic curve (implying a small, generally "accepting," a value). We can make similar predictions about the effect of the quotient ( Vcr— Vfa)/( Vca — Vm): a large value pushes the optimal threshold toward rejection (the steep part ofthe receiver operating characteristic curve), and a small value shifts it toward acceptance (the shallow part of the receiver operating characteristic curve). This result agrees with intuition because a large ( Vcr— Vfa)/( Vca— Vm) value means that the premium for correct be-

ideal point ideal point

Figure 2.3. An annotated receiver operating characteristic (ROC) curve. Signal detection theory gives the optimal behavior in terms of a critical likelihood ratio that we can visualize as the slope of the receiver operating characteristic (ROC). For example, if true states are rare, then we expect a high critical likelihood ratio that corresponds to a point on the steep portion of the receiver operating characteristic curve, as in point A. If, on the other hand, true states are common, then we expect a lower critical likelihood ratio that corresponds to a point on the shallower portion of the receiver operating characteristic curve, as in point B.

Figure 2.3. An annotated receiver operating characteristic (ROC) curve. Signal detection theory gives the optimal behavior in terms of a critical likelihood ratio that we can visualize as the slope of the receiver operating characteristic (ROC). For example, if true states are rare, then we expect a high critical likelihood ratio that corresponds to a point on the steep portion of the receiver operating characteristic curve, as in point A. If, on the other hand, true states are common, then we expect a lower critical likelihood ratio that corresponds to a point on the shallower portion of the receiver operating characteristic curve, as in point B.

havior is greater in the "false" state than in the "true" state (i.e., Vcr — Vfa Vca — Vm).

Was this article helpful?

0 0

Post a comment