Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
The Role of Learning Algorithms in Collective Action
Authors: Omri Ben-Dov, Jake Fawkes, Samira Samadi, Amartya Sanyal
ICML 2024 | Venue PDF | LLM Run Details | Input Tokens: 23,358 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,033 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical results, supported by a theoretical foundation, show that the effective size and success of the collective are highly dependent on properties of the learning algorithm. |
| Researcher Affiliation | Academia | 1Max Planck Institute for Intelligent Systems, T ubingen, Germany 2T ubingen AI Center 3Department of Statistics, University of Oxford. |
| Pseudocode | Yes | Algorithm 1 JTT training; Algorithm 2 Learning from Failure; Algorithm 3 Ideal Continuous Reweighting |
| Open Source Code | No | No explicit statement about releasing the authors' source code. |
| Open Datasets | Yes | synthetic 2D dataset and CIFAR-10 (Krizhevsky, 2009). Waterbirds dataset (Sagawa et al., 2020). MNIST-CIFAR dataset in Shah et al. (2020). |
| Dataset Splits | Yes | relying on performance on a validation set as a stopping criterion. When the collective can influence both the training and validation set, we show that this stopping criteria makes them particularly sensitive to the collective s size in the validation set. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU, GPU models, or memory) used for running experiments. |
| Software Dependencies | No | In all experiments we used the Py Torch ADAM optimizer with the default parameters, a learning rate of 5 10 4 and a batch size of 128. (Does not include version numbers for PyTorch or Adam). |
| Experiment Setup | Yes | For the 2D datasets, we used an MLP with layers sizes of [64, 32, 16, 2] with Re LU activations. For all image datasets we used the Res Net50 model. In all experiments we used the Py Torch ADAM optimizer with the default parameters, a learning rate of 5 10 4 and a batch size of 128. Each experiment was run multiple times with different random seeds, and in all figures the lines represent the means over the seeds, and the region around the lines is the 95% confidence interval according to Student s t-distribution. |