Am Montag, 7.4.2025 wird TUdatalib wegen geplanten Wartungsarbeiten am Speichersystem von 9:00 bis voraussichtlich 9:30 nur eingeschränkt nutzbar sein (kein Datenupload und Download) | Due to scheduled maintenance on the storage system, using TUdatalib will be limited on Monday, April 7 2025 from 9:00 to approx. 9:30 (no data upload or download)
Scene-Centric Unsupervised Panoptic Segmentation
Description
Unsupervised panoptic segmentation aims to partition an image into semantically meaningful regions and distinct object instances without training on manually annotated data. In contrast to prior work on unsupervised panoptic scene understanding, we eliminate the need for object-centric training data, enabling the unsupervised understanding of complex scenes. To that end, we present the first unsupervised panoptic method that directly trains on scene-centric imagery. In particular, we propose an approach to obtain high-resolution panoptic pseudo labels on complex scene-centric data combining visual representations, depth, and motion cues. Utilizing both pseudo-label training and a panoptic self-training strategy yields a novel approach that accurately predicts panoptic segmentation of complex scenes without requiring any human annotations. Our approach significantly improves panoptic quality, e.g., surpassing the recent state of the art in unsupervised panoptic segmentation on Cityscapes by 9.4% points in PQ. Acknowledgments: This project was partially supported by the European Research Council (ERC) Advanced Grant SIMULACRON, DFG project CR 250/26-1 "4D-YouTube", and GNI Project ``AICC''. This project has also received funding from the ERC under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 866008). Additionally, this work has further been co-funded by the LOEWE initiative (Hesse, Germany) within the emergenCITY center [LOEWE/1/12/519/03/05.001(0016)/72] and by the State of Hesse through the cluster project ``The Adaptive Mind (TAM)''. Christoph Reich is supported by the Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA) through the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research. License: Code, predictions, and checkpoints are released under the Apache-2.0 license, except for the ResNet-50 DINO backbone (dino_RN50_pretrain_d2_format.pkl), which is adapted from CutLER and published under the CC BY-NC-SA 4.0 license.
Subject
unsupervised panoptic segmentation;scene understanding;unsupervised scene understanding;unsupervised segmentation;unsupervised learningsed;panoptic segmentation;segmentation;computer visionDFG subject classification
4.43-05 Bild- und Sprachverarbeitung, Computergraphik und Visualisierung, Human Computer Interaction, Ubiquitous und Wearable ComputingRelated third party funded projects
EC/H2020 | 866008 | REDHMWK | III L6-519/03/05.001-(0016) | emergenCity - TP Roth
HMWK | 500/10.001-(00012) | TAM - TP Roth
Related Resources
- Is supplement to: arXiv:2504.01955
Collections
-
Segmentation [8]
The following license files are associated with this item:
Except where otherwise noted, this item's license is described as Apache License 2.0