From Analytic to Clinical Validation: Moving AI/ML into Practice

Artificial Intelligence (AI) is increasingly being used in a wide variety of medical imaging applications. Most of the focus, however, is on algorithm and scheme development, but this is only part of the picture. In order to have an impact on clinical decision making, workflow and patient care these AI tools must be evaluated using real-world cases and actual clinical providers that are expected to use them in routine care. The techniques used to conduct these types of studies are less well known in this field thus investigators need to be trained the proper study design and analysis methods. This course will cover basic principles, techniques, and process for validating models developed using artificial intelligence (AI)/machine learning (ML) techniques. The primary goal of this course is to help the audience understand and apply fundamental principles related to designing, executing, and interpreting model evaluation studies. The course will be organized around two parts: analytic validation and clinical validation. In the first half, the audience will be exposed to approaches for performing a technical validation of a prediction model, including different study designs, appropriate statistical tests, metrics, dataset considerations, and decision curve analysis. The second half will cover the process of undertaking clinical validation that would address real-world use of models, regulatory and deployment issues. Topics include workflow integration, prospective clinical trials, reader impact studies, and regulatory approvals. Examples will focus on imaging-related models that are drawn from literature and the instructors’ personal experiences in prognostic modeling, computer-aided diagnosis, and imaging biomarker development.
Course level is Intermediate and will include some Advanced level concepts.