Share Email Print

Proceedings Paper • new

Reader Disagreement Index: a better measure of overall review quality monitoring in an oncology trial compared to adjudication rate
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Purpose: Blinded Independent Central Review (BICR) is highly recommended by regulatory authorities for oncology registration trials. “Adjudication rate” provided by “Two Reviewers and Adjudicator Paradigm” of BICR has been part of reviewer performance metrics and trial efficacy. However, adjudication rate does not consider the adjudicator agreement or disagreement rate of a reviewer. We analyzed that Reader Disagreement Index (RDI) is a better measure than adjudication rate to monitor reviewer performance in BICR. Methods: BICR adjudication data from 3 different clinical trials including 10 board-certified radiologist reviewers using Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria was analyzed. RDI for each reviewer was calculated using the below mentioned formula. Reviewer adjudication rate and adjudicator agreement rate was calculated for each reviewer along with RDI. RDI was used to identify the discordant reviewer with highest disagreement rate. Number of cases where adjudicator disagreed with given reader RDI (%) = Total number of cases read ×100 Results: RDI identified the discordant reviewer in all 3 studies. Discordant reviewers identified using RDI were not the reviewers with highest adjudication or lowest agreement rates. Adjudication rate identified the discordant reviewer in 1 of the 3 studies. Reviewer with lowest adjudicator agreement could not have been identified as discordant reviewer using only adjudication rate in monitoring reviewer performance. RDI is more robust in identifying a discordant reviewer who neither has highest adjudication nor lowest agreement rate. Conclusions: RDI is more reliable measure of reviewer performance as compared to adjudication rate and could be effectively used to monitor reviewer performance as it combines both reviewer adjudication percentage and adjudication agreement percentage.

Paper Details

Date Published: 4 March 2019
PDF: 7 pages
Proc. SPIE 10952, Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment, 109520Q (4 March 2019); doi: 10.1117/12.2512611
Show Author Affiliations
Manish Sharma, PAREXEL International Corp. (India)
J. Michael O'Connor, PAREXEL International Corp. (United States)
Anitha Singareddy, PAREXEL International Corp. (India)

Published in SPIE Proceedings Vol. 10952:
Medical Imaging 2019: Image Perception, Observer Performance, and Technology Assessment
Robert M. Nishikawa; Frank W. Samuelson, Editor(s)

© SPIE. Terms of Use
Back to Top