Share Email Print

Proceedings Paper

Detection of frame informativeness in endoscopic videos using image quality and recurrent neural networks
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Gastroenterologists are estimated to misdiagnose up to 25% of esophageal adenocarcinomas in Barrett's Esophagus patients. This prompts the need for more sensitive and objective tools to aid clinicians with lesion detection. Artificial Intelligence (AI) can make examinations more objective and will therefore help to mitigate the observer dependency. Since these models are trained with good-quality endoscopic video frames to attain high efficacy, high-quality images are also needed for inference. Therefore, we aim to develop a framework that is able to distinguish good image quality by a-priori informativeness classification which leads to high inference robustness. We show that we can maintain informativeness over the temporal domain using recurrent neural networks, yielding a higher performance on non-informativeness detection compared to classifying individual images. Furthermore, it is also found that by using Gradient weighted Class Activation Map (Grad-CAM), we can better localize informativeness within a frame. We have developed a customized Resnet18 feature extractor with 3 classifiers, consisting of a Fully-Connected (FC), Long-Short-Term-Memory (LSTM) and a Gated-Recurrent-Unit (GRU) classifier. Experimental results are based on 4,349 frames from 20 pullback videos of the esophagus. Our results demonstrate that the algorithm achieves comparative performance with the current state-of-the-art. The FC and LSTM classifier reach an F1 score of 91% and 91%. We found that the LSTM classifier based Grad-CAMs represent the origin of non-informativeness the best as 85% of the images were found to be highlighting the correct area.

The benefit of our novel implementation for endoscopic informativeness classification is that it is trained end- to-end, incorporates the spatiotemporal domain in the decision making for robustness, and makes the model decisions of the model insightful with the use of Grad-CAMs.

Paper Details

Date Published: 10 March 2020
PDF: 6 pages
Proc. SPIE 11313, Medical Imaging 2020: Image Processing, 1131315 (10 March 2020); doi: 10.1117/12.2545734
Show Author Affiliations
T. G. W. Boers, Eindhoven Univ. of Technology (Netherlands)
J. van der Putten, Eindhoven Univ. of Technology (Netherlands)
J. de Groof, Amsterdam Univ. Medical Ctr. (Netherlands)
M. Struyvenberg, Amsterdam Univ. Medical Ctr. (Netherlands)
K. Fockens, Amsterdam Univ. Medical Ctr. (Netherlands)
W. Curvers, Catharina Hospital (Netherlands)
E. Schoon, Catharina Hospital (Netherlands)
F. van der Sommen, Eindhoven Univ. of Technology (Netherlands)
J. Bergman, Amsterdam Univ. Medical Ctr. (Netherlands)
P.H. N. de With, Eindhoven Univ. of Technology (Netherlands)

Published in SPIE Proceedings Vol. 11313:
Medical Imaging 2020: Image Processing
Ivana Išgum; Bennett A. Landman, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?