arXiv preprint ·
Discovering Semantic Granularity in Images through Clustering of Pretrained Visual Representations
- Published
- Source
- arXiv preprint, posted before peer review
- Original
- Read it at arXiv
- Topics
Abstract
Semantic granularity, which refers to the levels at which visual representations organize image content into meaningful entities, is fundamental to image understanding. However, whether pretrained visual representations inherently encode semantic structures at multiple granularities remains poorly understood. This paper presents a representation-driven approach for discovering semantically meaningful image regions at different granularities through clustering of pretrained visual representations without semantic supervision. Specifically, we concatenate selected feature maps extracted from a pretrained visual model and apply a singular value decomposition (SVD)-based projection to derive pixel-wise principal representation map tensor (PRMs). The resulting PRMs captures latent semantic structures embedded in pretrained visual representations, making them accessible to subsequent semantic clustering. We then introduce the semantic continuity rate (SCR), based on the silhouette principle, to quantify the clustering behavior of PRMs at different candidate granularities. Furthermore, we analyze the signal-to-noise ratio (SNR) of SCR profiles to assess the PRMs suitability for discovering semantic-like meaningful regions at multiple granularities. Our results indicate that PRMs exhibiting a higher proportion of relatively high SNR values over an initial range of cluster numbers tend to yield more semantically meaningful regions across multiple granularities. We evaluate the proposed approach on the COCO-2017 and VOC-2012 validation datasets using Hungarian-matched intersection over union (HIoU). The results demonstrate competitive segmentation performance among the compared methods, supporting the feasibility of discovering useful semantic granularities through clustering PRMs without semantic supervision.
The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.
Read it on ReadSearch
Each opens after you create an account, and brings you back to this paper.
More in Computer Vision
The DynaCell Evaluation Suite for Dynamic 3D Virtual Staining of Live Cells
CHROMA-MILL: Joint Low-Light Enhancement and White Balance under Colored Illumination
Findings-aware Language-Image Pretraining for CT Volume Interpretation
Is This 3D Asset Usable? Beyond the Beauty Render for Asset Curation
AgiWorld: An Agentic Video World Model for Building Persistent Worlds
Get the few papers worth your time in Computer Vision every morning.
ReadSearch reads the new papers in your field each day and brings you the few worth reading, each with its key finding in a sentence.