Vision-language models for radiology AI (Invited Paper)

Over 75% of all FDA-cleared software as a medical device relate to use cases in radiology. Despite this large prevalence, the current status-quo of training artificial intelligence (AI) tools entails using unimodal (imaging-only) algorithms. Moreover, retraining such models for new tasks requires using training supervised AI algorithms from scratch, using manually curated labels from scratch, even if it may be for the same modality or anatomy. In radiology, generating such labels requires expensive clinical expert time, limiting the development of capable AI models across tasks. In this presentation, I will describe the development and use of multi-modal vision-language models (VLMs) for radiological applications. VLMs present numerous benefits such as zero-shot classification, label-efficient adaptation to varying tasks, and improved robustness. Such new capabilities provided by VLMs is poised to usher in a new era of models for solving current and future challenges in radiology.