Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Criterion Collapse and Loss Distribution Control
Authors: Matthew J. Holland
ICML 2024 | Venue PDF | LLM Run Details | Input Tokens: 27,152 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,400 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We complement our basic theory with a set of experiments in 4, training non-linear neural network models (e.g., Res Net-34) for image classification from scratch, comparing across a variety of learning criteria with a common base loss. |
| Researcher Affiliation | Academia | 1SANKEN, Osaka University, Japan. Correspondence to: Matthew J. Holland <EMAIL>. |
| Pseudocode | No | The paper provides mathematical expressions for update rules (e.g., equation 16 and 17) but does not include structured pseudocode or algorithm blocks with labels like 'Algorithm' or 'Pseudocode'. |
| Open Source Code | Yes | A Git Hub repository with code and seed values to re-create all the results presented in this paper is available at this URL: https://github.com/feedbackward/collapse. |
| Open Datasets | Yes | We use four standard datasets: CIFAR-10, CIFAR-100, Fashion MNIST, and SVHN, accessed via the torchvision package. |
| Dataset Splits | Yes | In each trial, however, we shuffle the training data and do an 80-20 split for training-validation. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory, or cloud instance types) used for running the experiments. It only mentions model architectures. |
| Software Dependencies | No | The paper mentions using 'torchvision package' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | As an optimizer, we use vanilla SGD with step size 0.1 and momentum 0.9, run for 250 epochs, with mini-batch size of 200. |