Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
On the rankability of visual embeddings
Authors: Ankit Sonthalia, Arnas Uselis, Seong Joon Oh
NeurIPS 2025 | Venue PDF | LLM Run Details | Input Tokens: 34,289 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 3,546 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To address (1), we evaluate 7 modern visual encoders, from Res Net to CLIP, across 9 datasets with 7 attributes: age, crowd count, 3 head pose angles (pitch, roll, yaw), image aesthetics, and recency. We find that many embedding spaces are indeed rankable (Section 3). |
| Researcher Affiliation | Academia | Ankit Sonthalia University of Tübingen EMAIL Arnas Uselis University of Tübingen EMAIL Seong Joon Oh University of Tübingen |
| Pseudocode | No | The paper describes methods in prose, such as the definition of rankability and experimental procedures in Section 3 and 4, but does not include a distinct pseudocode or algorithm block. |
| Open Source Code | Yes | Our code is available at https://github.com/aktsonthalia/rankablevision-embeddings. |
| Open Datasets | Yes | In total, we use 9 datasets, covering 7 attributes. We provide a detailed breakdown in Table 1. ... UTKFace, introduced in [83], is a dataset of face images with age labels ranging from 0 to 116. ... The dataset was downloaded from the official website (https://susanqq.github.io/UTKFace/). ... UCF-QNRF, introduced in [7], is a large crowd counting dataset ... We use the official download link at https://www.crcv.ucf.edu/data/ucf-qnrf/ and the official train-test splits. |
| Dataset Splits | Yes | Table 1: Datasets and attributes. Summary of datasets used for evaluating the rankability of visual representations. ... Age Adience [11] 14k 4k 8 age groups Official 5-fold ... Crowd count UCF-QNRF [7] 1,201 334 49 12,865 Official ... Yaw BIWI Kinect 10,493 4,531 75 6 test seqs. ... We use the split provided by [78] in their official repository (https://github.com/uynaes/Ranking Aware CLIP/tree/main/examples). |
| Hardware Specification | Yes | All experiments were conducted on a single NVIDIA A100 GPU with 40GB of memory. |
| Software Dependencies | No | The paper mentions using the stuned Python library [55] and models obtained from timm [72], but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | Linear and nonlinear regression. We test 30 random hyperparameter configurations per datasetmodel pair. The initial learning rate is sampled from a log-uniform distribution over [10 6, 10 1] and decayed over a cosine schedule to zero, while the weight decay is sampled from a log-uniform distribution over [10 7, 10 4]. Data augmentation (horizontal flipping) is also toggled on or off randomly for a given run. We use 100 epochs and a batch size of 128 throughout. For nonlinear regression, we use a 2-layer MLP with 128 hidden dimensions and Re LU non-linearity. |