Comparing visual co-registration methods for UAV and satellite RGB imagery with semantic filtering of key points

Image-to-image correspondence is important in numerous remote sensing applications ranging from image mosaicking to 3D reconstruction. While many local features used for these methods aim for robustness to changes in viewpoint/illumination, recent studies have suggested that traditional feature extractors may lack stability in multi-temporal applications. We have discovered that this is especially true in multi-modal sensor contexts, such as corresponding high-resolution UAS images to broad area overhead imagery (e.g., satellite images). This paper explores the performance of various local feature extraction methods as they pertain to image-to-image correspondence in scenes captured at different times, with different sensors. Experiments here specifically evaluate co-registration between low-altitude, nadir UAV frames, and imagery collected from satellite sources. Due to challenges in the localization of imagery with significantly different resolutions, spatial extents, and spectral characteristics, two further studies are presented beyond baseline evaluation. First, images undergo histogram matching to better understand how the discussed algorithms’ performance changes as image characteristics become more or less similar. Secondly, experiments are performed where key point feature matches are refined with information taken from segmentation maps inferred by pre-trained segmentation models. These methods are evaluated in regions where satellite and UAV images have been collected at different times, with spatial correspondences being hand-labeled.