Boosting Unsupervised Semantic Segmentation with Principal Mask Proposals
![No Thumbnail [100%x80]](data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxvbmU9InllcyI/PjxzdmcgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB3aWR0aD0iMTI2NCIgaGVpZ2h0PSI4MCIgdmlld0JveD0iMCAwIDEyNjQgODAiIHByZXNlcnZlQXNwZWN0UmF0aW89Im5vbmUiPjwhLS0KU291cmNlIFVSTDogaG9sZGVyLmpzLzEwMCV4ODAvdGV4dDpObyBUaHVtYm5haWwKQ3JlYXRlZCB3aXRoIEhvbGRlci5qcyAyLjYuMC4KTGVhcm4gbW9yZSBhdCBodHRwOi8vaG9sZGVyanMuY29tCihjKSAyMDEyLTIwMTUgSXZhbiBNYWxvcGluc2t5IC0gaHR0cDovL2ltc2t5LmNvCi0tPjxkZWZzPjxzdHlsZSB0eXBlPSJ0ZXh0L2NzcyI+PCFbQ0RBVEFbI2hvbGRlcl8xOTYzNzlmM2I3OSB0ZXh0IHsgZmlsbDojQUFBQUFBO2ZvbnQtd2VpZ2h0OmJvbGQ7Zm9udC1mYW1pbHk6QXJpYWwsIEhlbHZldGljYSwgT3BlbiBTYW5zLCBzYW5zLXNlcmlmLCBtb25vc3BhY2U7Zm9udC1zaXplOjYzcHQgfSBdXT48L3N0eWxlPjwvZGVmcz48ZyBpZD0iaG9sZGVyXzE5NjM3OWYzYjc5Ij48cmVjdCB3aWR0aD0iMTI2NCIgaGVpZ2h0PSI4MCIgZmlsbD0iI0VFRUVFRSIvPjxnPjx0ZXh0IHg9IjM0MC41IiB5PSI2OS40Ij5ObyBUaHVtYm5haWw8L3RleHQ+PC9nPjwvZz48L3N2Zz4=)
Beschreibung
Unsupervised semantic segmentation aims to automatically partition images into semantically meaningful regions by identifying global semantic categories within an image corpus without any form of annotation. Building upon recent advances in self-supervised representation learning, we focus on how to leverage these large pre-trained models for the downstream task of unsupervised segmentation. We present PriMaPs – Principal Mask Proposals – decomposing images into semantically meaningful masks based on their feature representation. This allows us to realize unsupervised semantic segmentation by fitting class prototypes to PriMaPs with a stochastic expectation-maximization algorithm, PriMaPs-EM. Despite its conceptual simplicity, PriMaPs-EM leads to competitive results across various pre-trained backbone models, including DINO and DINOv2, and across different datasets, such as Cityscapes, COCO-Stuff, and Potsdam-3. Importantly, PriMaPs-EM is able to boost results when applied orthogonally to current state-of-the-art unsupervised semantic segmentation pipelines.
Zugehörige Drittmittelprojekte
EC/H2020 | 866008 | REDHMWK | 500/10.001-(00111) | 3AI - TP Roth
HMWK | 500/10.001-(00012) | TAM - TP Roth
HMWK | 500/10.001-(00111) | 3AI-NWG Schaub-Meyer
Verknüpfte Ressourcen
- Wird beschrieben durch: arXiv:https://arxiv.org/abs/2404.16818
- Wird beschrieben durch: https://openreview.net/forum?id=UawaTQzfwy
- Wird beschrieben durch: https://github.com/visinf/primaps
Sammlungen
-
Segmentation [9]
Die folgenden Lizenzbestimmungen sind mit dieser Ressource verbunden:
Solange nicht anders angezeigt, wird die Lizenz wie folgt beschrieben: Apache License 2.0