Scene-Centric Unsupervised Panoptic Segmentation
| datacite.relation.isSupplementTo | https://arxiv.org/abs/2504.01955 | |
| dc.contributor.author | Hahn, Oliver | |
| dc.contributor.author | Reich, Christoph | |
| dc.contributor.author | Araslanov, Nikita | |
| dc.contributor.author | Cremers, Daniel | |
| dc.contributor.author | Rupprecht, Christian | |
| dc.contributor.author | Roth, Stefan | |
| dc.date.accessioned | 2025-04-03T14:12:02Z | |
| dc.date.available | 2025-04-03T14:12:02Z | |
| dc.date.created | 2025-06-11 | |
| dc.date.issued | 2025-04-03 | |
| dc.description | Unsupervised panoptic segmentation aims to partition an image into semantically meaningful regions and distinct object instances without training on manually annotated data. In contrast to prior work on unsupervised panoptic scene understanding, we eliminate the need for object-centric training data, enabling the unsupervised understanding of complex scenes. To that end, we present the first unsupervised panoptic method that directly trains on scene-centric imagery. In particular, we propose an approach to obtain high-resolution panoptic pseudo labels on complex scene-centric data combining visual representations, depth, and motion cues. Utilizing both pseudo-label training and a panoptic self-training strategy yields a novel approach that accurately predicts panoptic segmentation of complex scenes without requiring any human annotations. Our approach significantly improves panoptic quality, e.g., surpassing the recent state of the art in unsupervised panoptic segmentation on Cityscapes by 9.4% points in PQ. Acknowledgments: This project was partially supported by the European Research Council (ERC) Advanced Grant SIMULACRON, DFG project CR 250/26-1 "4D-YouTube", and GNI Project ``AICC''. This project has also received funding from the ERC under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 866008). Additionally, this work has further been co-funded by the LOEWE initiative (Hesse, Germany) within the emergenCITY center [LOEWE/1/12/519/03/05.001(0016)/72] and by the State of Hesse through the cluster project ``The Adaptive Mind (TAM)''. Christoph Reich is supported by the Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA) through the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research. License: Code, predictions, and checkpoints are released under the Apache-2.0 license, except for the ResNet-50 DINO backbone (dino_RN50_pretrain_d2_format.pkl), which is adapted from CutLER and published under the CC BY-NC-SA 4.0 license. | de_DE |
| dc.identifier.uri | https://tudatalib.ulb.tu-darmstadt.de/handle/tudatalib/4532 | |
| dc.language.iso | en | de_DE |
| dc.rights.license | Apache-2.0 (https://www.apache.org/licenses/LICENSE-2.0) | |
| dc.subject | unsupervised panoptic segmentation | de_DE |
| dc.subject | scene understanding | de_DE |
| dc.subject | unsupervised scene understanding | de_DE |
| dc.subject | unsupervised segmentation | de_DE |
| dc.subject | unsupervised learningsed | de_DE |
| dc.subject | panoptic segmentation | de_DE |
| dc.subject | segmentation | de_DE |
| dc.subject | computer vision | de_DE |
| dc.subject.classification | 4.43-05 | |
| dc.subject.ddc | 004 | |
| dc.title | Scene-Centric Unsupervised Panoptic Segmentation | de_DE |
| dc.type | Software | de_DE |
| dcterms.accessRights | openAccess | |
| person.identifier.orcid | 0009-0008-6164-1035 | |
| person.identifier.orcid | 0000-0002-8616-1627 | |
| person.identifier.orcid | 0000-0002-9424-8837 | |
| person.identifier.orcid | 0000-0002-3079-7984 | |
| person.identifier.orcid | #PLACEHOLDER_PARENT_METADATA_VALUE# | |
| person.identifier.orcid | 0000-0001-9002-9832 | |
| tuda.project | EC/H2020 | 866008 | RED | |
| tuda.project | HMWK | III L6-519/03/05.001-(0016) | emergenCity - TP Roth | |
| tuda.project | HMWK | 500/10.001-(00012) | TAM - TP Roth | |
| tuda.unit | TUDa |
Files
Original bundle
1 - 6 of 6
| Name | Description | Size | Format | |
|---|---|---|---|---|
| raft_smurf.pt | Model parameters | 20.07 MB | Unknown data format | |
| dino_RN50_pretrain_d2_format.pkl | Model parameters | 89.9 MB | Unknown data format | |
| depthg.ckpt | Model parameters | 336.3 MB | Unknown data format | |
| cups_code.zip | Training and inference code (PyTorch) | 27.67 MB | ZIP-Archivdateien | |
| cups_cityscapes_predictions.zip | Inference results | 55.65 MB | ZIP-Archivdateien | |
| cups.ckpt | Model parameters | 916.15 MB | Unknown data format |