Brevin Tilmon, PhD Student at University of Florida, and Dr. Sanjeev Koppal, Associate Professor in the Department of Electrical and Computer Engineering, are researching adaptive resolution, which allows cameras to distribute higher resolution onto parts of images that matter most. While most other efforts on this topic apply attention within network training and testing after images have been captured, the SaccadeCam framework leverages visual attention to distribute resolution during image capture. For example, this technology can help detect important objects for robot navigation, such as poles, signs and distant vehicles.
Their published paper, “SaccadeCam: Adaptive Visual Attention for Monocular Depth Sensing,” was accepted to IEEE/CVF International Conference on Computer Vision (ICCV) in 2021.
Abstract:
Most monocular depth sensing methods use conventionally captured images that are created without considering scene content. In contrast, animal eyes have fast mechanical motions, called saccades, that control how the scene is imaged by the fovea, where resolution is highest. In this paper, we present the SaccadeCam framework for adaptively distributing resolution onto regions of interest in the scene. Our algorithm for adaptive resolution is a self-supervised network and we demonstrate results for end-to-end learning for monocular depth estimation. We also show preliminary results with a real SaccadeCam hardware prototype.