Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Unsupervised Learning under Latent Label Shift
Authors: Manley Roberts, Pranav Mani, Saurabh Garg, Zachary Lipton
NeurIPS 2022 | Venue PDF | LLM Run Details | Input Tokens: 22,572 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 6,334 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In semi-synthetic experiments, we adapt existing image classification benchmarks to the LLS setting, sampling without replacement to construct collections of label-shifted domains. and Table 1: Results on CIFAR-20. Each entry is produced with the averaged result of 3 different random seeds. |
| Researcher Affiliation | Academia | Manley Roberts Pranav Mani Saurabh Garg Zachary C. Lipton Carnegie Mellon University EMAIL; EMAIL |
| Pseudocode | Yes | Algorithm 1 DDFA Training and Algorithm 2 DDFA Prediction |
| Open Source Code | Yes | Code is available at https://github.com/acmi-lab/Latent-Label-Shift-DDFA. |
| Open Datasets | Yes | First we examine standard multiclass image datasets CIFAR-10, CIFAR-20 [43], and Image Net-50 [19] containing images from 10, 20, and 50 classes respectively. |
| Dataset Splits | Yes | Algorithm 1 DDFA Training: 1: Split into train set T and validation set V and The algorithm uses train and validation data consisting of pairs of images and domain indices. We train Res Net50 [35] (with added dropout) on images xi with domain indices di as the label, choose best iteration by valid loss, pass all training and validation data through pf, and cluster pushforward predictions pfpxiq into m ě k clusters with Faiss K-Means [42]. |
| Hardware Specification | Yes | We use one NVIDIA V100 GPU for all experiments. |
| Software Dependencies | No | The paper mentions several software components and libraries, such as 'Res Net50', 'Sim CLR', 'Mo Co', 'Faiss K-Means', and 'Sklearn implementation of standard NMF', but it does not specify any version numbers for these dependencies. |
| Experiment Setup | Yes | We train for 100 epochs with a learning rate of 1e-4, batch size of 256, and Adam optimizer. We use a learning rate of 1e-3, batch size of 256 for SCAN. |