Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Model
![No Thumbnail [100%x80]](data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9InllcyI/PjxzdmcgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB3aWR0aD0iMTI2NCIgaGVpZ2h0PSI4MCIgdmlld0JveD0iMCAwIDEyNjQgODAiIHByZXNlcnZlQXNwZWN0UmF0aW89Im5vbmUiPjwhLS0KU291cmNlIFVSTDogaG9sZGVyLmpzLzEwMCV4ODAvdGV4dDpObyBUaHVtYm5haWwKQ3JlYXRlZCB3aXRoIEhvbGRlci5qcyAyLjYuMC4KTGVhcm4gbW9yZSBhdCBodHRwOi8vaG9sZGVyanMuY29tCihjKSAyMDEyLTIwMTUgSXZhbiBNYWxvcGluc2t5IC0gaHR0cDovL2ltc2t5LmNvCi0tPjxkZWZzPjxzdHlsZSB0eXBlPSJ0ZXh0L2NzcyI+PCFbQ0RBVEFbI2hvbGRlcl8xOTY1YjM4YjdiMCB0ZXh0IHsgZmlsbDojQUFBQUFBO2ZvbnQtd2VpZ2h0OmJvbGQ7Zm9udC1mYW1pbHk6QXJpYWwsIEhlbHZldGljYSwgT3BlbiBTYW5zLCBzYW5zLXNlcmlmLCBtb25vc3BhY2U7Zm9udC1zaXplOjYzcHQgfSBdXT48L3N0eWxlPjwvZGVmcz48ZyBpZD0iaG9sZGVyXzE5NjViMzhiN2IwIj48cmVjdCB3aWR0aD0iMTI2NCIgaGVpZ2h0PSI4MCIgZmlsbD0iI0VFRUVFRSIvPjxnPjx0ZXh0IHg9IjM0MC41IiB5PSI2OS40Ij5ObyBUaHVtYm5haWw8L3RleHQ+PC9nPjwvZz48L3N2Zz4=)
Beschreibung
Omnidirectional depth perception is essential for mobile robotics applications that require scene understanding across a full 360° field of view. Camera-based setups offer a cost-effective option by using stereo depth estimation to generate dense, high-resolution depth maps without relying on expensive active sensing. However, existing omnidirectional stereo matching approaches achieve only limited depth accuracy across diverse environments, depth ranges, and lighting conditions, due to the scarcity of real-world data. We present DFI-OmniStereo, a novel omnidirectional stereo matching method that leverages a large-scale pre-trained foundation model for relative monocular depth estimation within an iterative optimization-based stereo matching architecture. We introduce a dedicated two-stage training strategy to utilize the relative monocular depth features for our omnidirectional stereo matching before scale-invariant fine-tuning. DFI-OmniStereo achieves state-of-the-art results on the real-world Helvipad dataset, reducing disparity MAE by approximately 16% compared to the previous best omnidirectional stereo method.
Schlagwort
Depth Prediction;Omnidirectional depth perception;Stereo Matching;deep learning;foundation modelDFG-Fächer
4.43-04 Künstliche Intelligenz und Maschinelle Lernverfahren4.43-05 Bild- und Sprachverarbeitung, Computergraphik und Visualisierung, Human Computer Interaction, Ubiquitous und Wearable Computing
Zugehörige Drittmittelprojekte
EC/H2020 | 866008 | REDHMWK | III L6-519/03/05.001-(0016) | emergenCity - TP Roth
HMWK | 500/10.001-(00012) | TAM - TP Roth
Verknüpfte Ressourcen
- Wird beschrieben durch: arXiv:2503.23502
Sammlungen
-
Segmentation [9]
Die folgenden Lizenzbestimmungen sind mit dieser Ressource verbunden:
Solange nicht anders angezeigt, wird die Lizenz wie folgt beschrieben: Apache License 2.0