I will present a deep learning framework for content-aware estimation of tissue optical properties from wide-field images. Spatial frequency domain imaging is used to acquire ground-truth measurements of scattering and absorption coefficients of a variety of tissues. A generative network is then adversarially trained to estimate these properties from new tissues directly from unstructured or structured light. This data-driven approach has some advantages in accuracy and speed compared to model-based approaches.

Imaging the heterogeneity of the tumor microenvironment is thought to be valuable for assessing the efficacy of a cancer therapy. However, due to the insufficient resolution of clinically available molecular imaging tools such as positron emission tomography, this feat is currently not achievable. Here, we demonstrate that revealing the TME is possible with immuno-near-infrared-fluorescence (Immuno-NIRF) imaging. Moreover, we combined immuno-NIRF with optical coherence tomography (OCT) to provide structural context to the molecular images. The combination of the two techniques may represent a novel tool in the operating room for assessing development or early diagnosis of tumors.

Adequate tumor margin delineation is crucial to maximize positive patient outcomes in molecular-guided surgery. Raman spectroscopy is highly specific in detecting tumor margins based on the differences in molecular composition between tumor and normal tissue; however, one major technical hurdle to its adoption is its slow acquisition speed. Previously, we described a "superpixel" acquisition approach that can expedite up to 10,000x compared to point-bypoint scanning while covering the entire surface area. We detected human basal cell carcinoma in Mohs surgical resection margins from eight patients and demonstrated superpixel acquisition had consistent diagnostic performance with point-by-point scanning. In this work, we further demonstrated examples of raster-scanned superpixel Raman classification images of positive and negative margins from three new patients. The performance of three superpixel sizes were evaluated, including 25×25μm², 50×50μm² and 100×100μm². A previous established biophysical inverse model was applied to extract the biochemical composition of each superpixel, and a prior classification model was employed to generate the tumor heatmap. The classification result was then compared with the histopathological image. Our results show that superpixel Raman imaging can overcome the limitation of traditional Raman imaging in speed, allowing for rapid tumor margin assessment.

In this project, we analyzed 30 healthy and tumorous breast samples using static and dynamic full field optical coherence tomography (FF-OCT). We developed an automatic analysis workflow to classify each sample and compared it to an independent standard histological diagnosis. We used a first machine-learning algorithm to obtain cell and fiber segmentation of FF-OCT images and applied a linear support vector machine (SVM) analysis to classify each sample. We could obtain 100% specificity and sensitivity compared to histology. The label-free and non-invasive combination of static and dynamic FF-OCT thus appears very promising to obtain an efficient diagnosis of tumoral samples.

Hyperspectral imaging (HSI) enables visualisation of a range of morphological and biochemical information from tissue, which could improve disease diagnostic accuracy. Unfortunately, HSI presents a substantial instrumentation challenge for biomedical applications, particularly in endoscopy where a wide range of image distortions arise that can lead to artefacts in the 3D HSI data. To address this challenge, we have developed several methods that enable the addition of HSI to endoscopic imaging, encompassing both novel sensor development, application of geometric image co-registration methods, and development of machine learning tools for rapid real-time HSI data analysis. In this presentation, I will give an overview of these advances, including our latest results from clinical studies applying HSI endoscopy in the gastrointestinal tract for early cancer detection.

This work proposes separating data analysis from hyperspectral enhancement or editing, providing a robust, context-independent, fully-tunable framework for biomarker-based contrast in wide-field imaging with a series of reliable properties that could enable its use in guided surgery. Some applications of this method powered by deep learning diagnostics will be discussed and shown.

Near-infrared fluorescence image-guided surgery offers substantial benefits for both surgeons and patients, but high-performance near-infrared fluorescent molecular markers are not available for clinical application. To enable adoption of a wider variety of fluorescent markers, we have developed a single-chip snapshot multispectral imaging system that combines a scientific CMOS image sensor with high quantum efficiency (94%), low read noise (1.6 e-), and high dynamic range (91 dB) and pixelated interference filters that enable color/near-infrared imaging without co-registration error The fully integrated imaging system is currently being evaluated for tumor detection with targeted probes and sentinel lymph node detection with indocyanine green.

The cellular microenvironment in disease models is increasingly being recognized as a key contributing factor in disease onset and progression. Particularly in cancer, key features of the cellular microenvironment such as metabolic fluxes and organization of the collagen rich extracellular matrix (ECM) have been demonstrated to be candidate image based biomarkers for cancer invasion and progression. However despite the great promise of these microenvironment image features, their application to classic wound models has been limited for several reasons including a lack of appropriate methods and instrumentation for clinical wound application. We will overview our collaborative work to quantitate metabolism and ECM organization in a range of wound models all using a combination of both intrinsic and extrinsic multiparametric optical signals. These signals includes polarization, fluorescence intensity, spectra and lifetime. We will discuss technical approaches and advances for each and early efforts to extend these to Fluorescence image-guided surgery (FIGS) application.

Early stages and the margins of disease can be invisible to human vision during surgical interventions. Molecular probes with fluorescence labeling are used to enhance contrast of lesions to distinguish their location and extent. However, low intensity fluorescence signals are problematic because photobleaching permanently loses this critical information. By using low power illumination from a scanning fiber endoscope (SFE), the rate of information loss is reduced by over 6x in glioma mouse model using 5ALA-PpIX compared to a clinical surgical microscope. We propose using one or more SFEs as headlights near the tip of surgical tools to guide interventions.

Fluorescence imaging for surgical guidance is a proven modality that allows for visualization of fluorescent markers in numerous biological imaging applications. As the field continues to develop there is an urgent need for fluorescence-imaging standards that enable system characterization and performance monitoring. Here, we present a long-term stable standard that mimics indocynine green (ICG) spectral behavior and is capable of probing the sensitivity of commercially available imaging systems. The standard is composed of three different tests: a varying concentration sensitivity test (0.1nM-3000nM), a tissue-equivalent-depth sensitivity test (0.2mm-7mm), and a 1951 USFA fluorescence resolution test. The fluorescence standard platform developed can incorporate 3-D printing with tissue-equivalent layers, enabling a wide variety of anatomical-specific designs. The manufacturing capabilities developed allow for incorporation of different fluorophores for applications beyond ICG and NIR.

Current intraoperative methods to assess tumor invasion depth in mucosal oral cancer provide limited real-time information. The advent of targeted fluorescence contrast agents for head and neck cancer is a promising innovation, but surgical imaging systems typically provide only two-dimensional views. Here, we investigate the use of an image-guided fluorescence tomography (igFT) system to estimate the depth of tumor invasion in tissue-simulating oral cancer phantoms. Implementation of non-contact diffuse optical tomography using finite-element software (NIRFAST) is enabled with geometric data from intraoperative cone-beam CT (CBCT) imaging and surgical navigation. The tissue phantoms used gelatin for the background (5% for fat, 10% for muscle) and 2% agar for palpable, tumor-like inclusions. Standard agents were used for absorption (hemoglobin), scattering (Intralipid), fluorescence (indocyanine green), and CT contrast (iohexol). The agar inclusions were formed using 3D printed molds, and positioned at the surface of the gelatin background to mimic mucosal tumor invasion (an “iceberg” model). Simulations and phantom experiments characterize fluorescence tomography performance across a range of tumor invasion depths. To aid surgical visualization, the fluorescence volume is converted to a colored surface indicating tumor depth, and overlaid on the navigated endoscopic video. Clinical studies are necessary to assess in vivo performance and intraoperative workflow.

Due to the lack of objectively measurable or quantifiable methods to assess the bone perfusion, the success of removing devitalized bone is based almost entirely on surgeon’s experience and varies widely across surgeons and centers. In this study, an indocyanine green (ICG)-based dynamic contrast-enhanced fluorescence imaging (DCE-FI) has been developed to objectively assess bone perfusion and guide surgical debridement. A porcine trauma model (n = 6 pigs x 2 legs) with up to 5 conditions of severity in loss of flow in each, was imaged by a commercial fluorescence imaging system. By applying the bone-specific hybrid plug-compartment (HyPC) kinetic model to four-minute video sequences, the perfusion-related metrics, such as peak intensity, total bone blood flow (TBBF) and endosteal bone blood flow to TBBF fraction (EFF) were calculated. The results shown that the combination of TBBF and EFF can effectively differentiate injured from normal bone with the accuracy, sensitivity and specificity of 89%, 88% and 90%, respectively. Our subsequent first inhuman bone blood flow imaging study confirmed DCE-FI can be successfully translated into human orthopaedic trauma patients.

Imaging techniques for highly specific detection of cancer cells in-vivo can have applications ranging from preclinical drug discovery studies to clinical cancer diagnosis and surgical therapy. While fluorescence imaging using cancer-targeted antibodies has shown promise, non-specific probe accumulation in tissue results in a significant background fluorescence, reducing detection sensitivity using traditional intensity-based continuous wave (CW) fluorescence imaging. Here we demonstrate that fluorescence lifetime imaging can provide significant tumor contrast enhancement over CW intensity, with the use of Indocyanine Green and anti-epidermal growth factor receptor (EGFR) antibody conjugated to the fluorescent dye IRDye800CW.

Accidental nerve damage or transection of vital nerve structures remains an unfortunate reality that is often associated with surgery. Despite the existence of nerve-sparing techniques, the success of such procedures is not only complicated by anatomical variance across patients but is also highly dependent on a surgeon’s first-hand experience that is acquired over numerous procedures through trial and error, often with highly variable success rates. Fluorescent small molecules, such as rhodamines and fluoresceins have proven incredibly useful for biological imaging in the life sciences, and they appeared to have potential in illuminating vital nerve structures during surgical procedures. In order to make use of the current clinically relevant imaging systems and to provide surgeons with fluorescent contrast largely free from the interference of hemoglobin and water, it was first necessary to spectrally tune known fluorescent scaffolds towards near infrared (NIR) wavelengths. To determine whether the well-documented Si-substitution strategy could be applied towards developing a NIR fluorophore that retained nerve-specific properties of candidate molecules, an in vivo comparison was made between two compounds previously shown to highlight nervous structures – TMR and Rhodamine B – and their Si-substituted derivatives.

In the past several decades, a number of novel fluorescence image-guided surgery (FGS) contrast agents have been under development, with many in clinical translation and undergoing clinical trials. In this review, we have identified and summarized the contrast agents currently undergoing clinical translation. In total, 39 novel FGS contrast agents are being studied in 85 clinical trials. Four FGS contrast agents are currently being studied in phase III clinical trials and are poised to reach FDA approval within the next two to three years. Among all novel FGS contrast agents, a wide variety of probe types, targeting mechanisms, and fluorescence properties exists. Clinically available FGS imaging systems have been developed for FDA approved FGS contrast agents, and thus further clinical development is required to yield FGS imaging systems tuned for the variety of contrast agents in the clinical pipeline. Additionally, study of current FGS contrast agents for additional disease types and development of anatomy specific contrast agents is required to provide surgeons FGS tools for all surgical specialties and associated comorbidities. The work reviewed here represents a significant effort from many groups and further development of this promising technology will have an enormous impact on surgical outcomes across all specialties.

An appropriate identification of the tissue type that is in front of the excision surgical tool is critical, for either tumoral tissue removal, or even accessory healthy tissues whose destruction could cause severe collateral damage. Healthy tissue distinction is an unsolved task in real surgical praxis, due for instance to alterations or hemorrhages. Many approaches have tried to solve the problem of healthy tissue discrimination, such as spectroscopy. Diffuse Reflectance Spectroscopy (DRS), an easy to implement and reliable optical diagnostic technique, has shown great potential in tumoral tissue discrimination, as long as in healthy tissue distinction. However, the identification of nerve tissue in particular is challenging. What is more, nerve tissue distinction is critical in surgical interventions, due to the severe irreversible dysfunctions in patient mobility that collateral damage could cause. Fluorescence is an optical technique that could be of relevance for nerve identification. The need of specific fluorophores that tend to accumulate preferentially on nerve tissue is essential for this task. Image quality requires also an analysis of the optical source, filters employed, fluorophore concentration or inoculation method. Together with those aspects, additional open questions include incubation time or cytotoxicity, critical for clinical translation. In this work, fluorescence imaging for nerve identification is applied on in vivo rats. The approach uses several fluorophore concentrations, based on Oxazine, and combines topical and intravascular inoculation. Optical irradiance is considered for image contrast. The results show an appreciable nerve contrast for optimized parameters.

While the use of non-targeted dyes may be useful for fluorescence-guided surgery (FGS) in certain diseases, specific contrast agents are essential in oncology. As shown in the latest published clinical studies, monoclonal antibodies display all the characteristics that are necessary to play a major role in this field of medical imaging provided the antigenic target is relevant. Based on clinical data from FGS and from antibody expertise, major parameters explaining why antibodies are favorite FGS tools in oncology will be discussed as well as the remaining challenges for their development.

Pancreatic cancer is the third leading cause of cancer-related deaths in the United States. Up to 20% of patients are candidates for resection, which is currently the only potentially curative treatment option. However, pancreatic cancer patients have high rates of recurrence after resection due in part to cells left behind at the surgical margin. Fluorescence guided surgery (FGS) has emerged as a method to improve surgical resection. For pancreatic cancer patients eligible for surgical resection, intraoperative imaging would serve two purposes. For patients with resectable disease, image-guidance could help delineate tumor tissue to achieve complete resection. Of critical value, fluorescence guided surgery (FGS) could be used to identify locally metastatic disease, which could spare the patient from major, unnecessary surgery and move directly to other treatments. Recent studies demonstrated that MUC16 is overexpressed in 60-80% of pancreatic cancers, but this biomarker has not yet been explored for FGS of pancreatic cancer. Herein, we describe the validation and development of a near infrared fluorescence antibody that recognizes MUC16 for surgical imaging.

During resection of head and neck cancers, surgeons collect multiple small tissue samples from the excision wound bed for “frozen section” analysis by a clinical pathologist to determine clear margins. However, this process is disruptive to work flow, time-consuming, and expensive. We have validated Paired-Agent Imaging (PAI) in in vivo xenograft models using epidermal growth factor receptor and demonstrated that PAI exhibits higher diagnostic accuracy targeted agent alone – tested in both fresh tissue (~3-mm) and frozen sections (10-m). Here PAI will be extended to screening of tissue biopsies to quantify tumor burden in comparison to standard frozen section pathology.

Fluorescent molecular-guided surgery (FGS) is at a tipping point in terms of clinical approval and adoption in a number cancer applications, with ongoing phase 0 and phase 1 clinical trials being carried out in a wide range of cancers using a wide range of agents. The pharmacokinetics of each of these agents and the physiology of these cancers can differ vastly on a patient-to-patient basis, bringing to question: how can one fairly compare different methodologies (defined as the combination of imaging agent, system, and protocol) and how can existing methodologies be further optimized? To this point, little methodology comparison has been carried out, and the majority of FGS optimization has concerned system development—on the level of maximizing signal-to-noise, dynamic detection range, and sensitivity—independently from traditional agent development—in terms of fluorophore brightness, toxicity, solubility, and binding affinity and specificity. Here we propose an inclusion of tumor and healthy tissue physiology (blood flow, vascular permeability, specific and nonspecific binding sites, extracellular matrix, interstitial pressure, etc.) variability into the optimization process and re-establish well-described task-based metrics for methodology optimization and comparing quality of one methodology to another. Two salient conclusions were identified: (1) contrast-tobackground variability is a simple metric that correlates with difficult-to-carry-out task-based metrics for comparing methodologies, and (2) paired-agent imaging protocols offer unique advantages over singleimaging- agent studies for mitigating confounding tumor and background physiology variability.

Introduction: The clinical need for improved intraoperative tumor visualization has led to the development of targeted contrast agents for fluorescence-guided surgery (FGS). A key characteristic of these agents is their high tumor specificity, which could enable detection of residual lesions that would likely be missed by visual inspection. Here, we examine the utility of a promising somatostatin receptor subtype-2 (SSTR2)-targeted fluorescent agent for detecting residual disease in mouse xenografts using FGS and post-operative histopathological validation.

Methods: Mice (n=2) implanted with SSTR2 overexpressing tumors were injected with 2 nmol of the dual-labeled somatostatin analog, ⁶⁷Ga-MMC(IR800)-TOC, and tumors were resected 48 h post-injection using traditional white light reflectance and palpation. Tumors underwent gamma counting and histopathology analysis. The wide-field FGS imaging platform (OnLume) was used to evaluate residual disease in situ under ambient light representative of an operating room.

Results: The tumor was resected with grossly negative margins using conventional inspection and palpation; however, additional in situ residual disease was found in the tumor cavity using FGS imaging. In situ fluorescent tumor contrast-to-noise ratios (CNRs) were 3.0 and 5.2. Agent accumulation was 7.72 and 8.20 %ID/g in tumors and 0.27 and 0.20 %ID/g in muscle. Fluorescence pixel values and gamma counts were highly correlated (r = 0.95, P < 0.048). H&E and IHC staining confirmed cancer positivity and SSTR2-overexpression, respectively.

Conclusion: Our findings demonstrate that the use of clinically relevant fluorescence imaging instrumentation enhances the evaluation of promising FGS agents for in situ visualization of residual disease

Near-infrared (NIR) fluorescence imaging is a promising intraoperative technique for real-time visualization of tumor tissue during surgery. The process of clinical translation of novel fluorescent agents is an essential part in the evolution of NIR fluorescence guided surgery. Poor visualization of tumors during surgery is one of the major challenges surgeons often face in oncologic patients, mainly due to the improved neo-adjuvant treatment patients receive. In these cases, NIR fluorescence imaging with the use of tumor-targeted fluorescent agents can play an essential role and help provide better oncologic results or patient outcomes. However, before this technique can be implemented in standard of care, optimal tumor-targeted fluorescent agents need to be developed and novel fluorescent agents need to undergo a successful process of clinical translation. Here we describe the clinical translation of SGM-101, a fluorescent anti-CEA monoclonal antibody.

The large growth in fluorescence guided surgical (FGS) imaging have radically different physical designs, image processing approaches and performance requirements, making it nearly impossible to specify uniform performance goals. Yet, utilization of different devices in clinical trials indicates some need for common knowledge bases and quality assessment goals to ensure accurate and effective translation occurs. This task group identified fundamental image quality characteristics corresponding objective test methods that should be determined across a variety of FGS devices. This report outlines a cohort of explicit test methods and specific tissue simulating phantom tests, with the goal of suggested guidelines that will allow evaluative and comparative studies on system performance. This initiative is expected to help users and developers focus on needs while converging towards future goals.

Short-wave infrared (SWIR/NIR-II) fluorescence imaging has received increased attention for use in fluorescence-guided surgery (FGS) due to the potential for higher resolution imaging of subsurface structures and reduced autofluorescence compared to conventional NIR-I imaging. As with any fluorescence imaging modality introduced in the operating room, an appropriate accounting of contaminating background signal from other light sources in the operating room is an important step. Herein, we report the background signals in the SWIR and NIR-I emitted from commonly-used equipment in the OR, such as ambient and operating lights, LCD screens and surgical guidance systems. These results can guide implementation of protocols to reduce background signal.

Intraoperative assessment of resection completeness remains challenging in oncological head and neck surgery as is illustrated by 20-30% of inadequate resection margins at final pathological assessment. Inadequate surgical margins correlate with a significantly worse overall survival and warrant additional treatments, such as radio- and chemotherapy, which often result in patient morbidity and increased healthcare costs. Fluorescence-guided surgery, which is based on intraoperative visualization of a systemic infused fluorescence-labeled tumor tracer, is being developed to improve visualization of the tumor borders while operating. This technology would add important information to traditional tactile and visual data. The following chapter discusses the clinical application of fluorescence-guided surgery techniques, that can assist the surgeon in achieving oncologically sound head and neck cancer resection.

ABY-029, an anti-epidermal growth factor receptor (EGFR) Affibody molecule labeled with IRDye 800CW, has been used in three Phase 0 microdosing clinical trials for fluorescence guided surgery. In May of 2019, the clinical trials were put on hold because the ABY-029 produced under Good Manufacturing Practices (GMP) for human administration had come to the end of term in which the drug product was known to be stable. Stability testing was halted due to limitations in supply of a suitable reference standard and a required test product being discontinued from commercial sale. In order to complete the remaining patients in the three clinical trials, new stability tests were developed and the GMP batch of ABY- 029 drug product tested under the new protocols. The GMP batch of ABY-029 passed all stability tests under the new protocols and the Federal Drug Administration (FDA) has given permission to complete the remaining patients with stability testing of ABY-029 performed for each patient. The tests developed and used to test ABY-029 drug product stability are described here.

Surgical excision via wide local excision (WLE) of the primary sarcoma tumor is a mainstay of treatment due to the limited effectiveness of chemotherapy and radiation. Even with attempts at WLE, 22-34% of the patient will be diagnosed with a positive margin by the pathologist, necessitating additional radiation or surgery. Recent studies have demonstrated reduced local recurrence when using fluorescence-guided surgery (FGS) to detect residual sarcoma following attempted WLE.

ABY-029 is an anti-EGFR Affibody molecule labeled with IRDye800CW that is currently under Phase 0 human trial for FGS. To date, several studies have been performed to evaluate ABY-029 signal intensity in untreated human sarcoma xenografts; however, many patients undergoing cancer surgery have received pre-operative radiation and/or chemotherapy, which can affect tissue properties and tumor molecule expression level. Determining the effects of radiation and chemotherapy exposure on fluorophore binding in sarcomas may influence best practices in implementing FGS for sarcoma.

In this project, fluorophore signal intensities in tumor and surrounding tissue were measured and compared to the receptor concentration determined by immunohistochemistry. Here, we report the result for one EGFR positive synovial sarcoma cell lines, SW982. Four groups of human dose equivalent therapies – control, radiation, chemotherapy (Doxorubicin) and radiation followed by chemotherapy – were given to the tumor-bearing mice. The difference between groups can be used to determine the effects of preoperative sarcoma therapies on EGFR expression, ABY-029 uptake, and optical properties of tissues.