Share Email Print

Proceedings Paper

Efficiently calculating ROC curves, AUC, and uncertainty from 2AFC studies with finite samples
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Two-alternative forced-choice (2AFC) reader studies are useful for evaluating medical imaging devices because humans can rapidly make direct comparisons with high precision leading to low variability in study results. We propose a method for estimating the receiver operating characteristic (ROC) curve, reader performance (area under the ROC curve, AUC), and uncertainty on AUC from a series of 2AFC trials on a finite data set. Our method greatly reduces the number of 2AFC comparisons required by using an algorithm created for sorting, in this case Merge Sort. By altering the algorithm to work in discrete layers, we can make unbiased estimates as the study proceeds. Because the merging is pre-planned with a tree structure, we can use a Hanley-McNeil approximation to predict the reduction in variance in AUC from performing more 2AFC comparisons. The algorithm is also altered to increase the amount of time between the reader seeing the same image repeatedly thus decreasing potential learning. We compare our method with that of Massanes and Brankov (2016).

Paper Details

Date Published: 16 March 2020
PDF: 8 pages
Proc. SPIE 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment, 113160L (16 March 2020); doi: 10.1117/12.2550601
Show Author Affiliations
Dylan H. Shekter, Univ. of Maryland, College Park (United States)
Frank W. Samuelson, US Food and Drug Administration (United States)

Published in SPIE Proceedings Vol. 11316:
Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment
Frank W. Samuelson; Sian Taylor-Phillips, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?