Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
On the Spectral Bias of Neural Networks
Authors: Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, Aaron Courville
ICML 2019 | Venue PDF | LLM Run Details | Input Tokens: 17,049 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 7,078 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We find empirical evidence of a spectral bias: i.e. lower frequencies are learned first. We also show that lower frequencies are more robust to random perturbations of the network parameters (Section 3). We now present experiments showing that networks tend to fit lower frequencies first during training. |
| Researcher Affiliation | Academia | 1Mila, Quebec, Canada 2Image Analysis and Learning Lab, Ruprecht-Karls-Universit at Heidelberg, Germany. |
| Pseudocode | No | The paper presents theoretical derivations and experimental setups but does not include any clearly labeled 'Algorithm' or 'Pseudocode' blocks. |
| Open Source Code | Yes | 1Code: https://github.com/nasimrahaman/SpectralBias |
| Open Datasets | Yes | To tackle this, we propose the following set of experiments to measure the effect of spectral bias indirectly on MNIST. |
| Dataset Splits | No | The paper mentions 'validation loss curves' and states 'the validation set is obtained by evaluating 0 on a separate subset of the data', but does not provide specific details on the split percentages, sample counts, or the methodology for creating this subset for reproducibility. |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware used for running experiments, such as CPU or GPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions 'Pytorch s BCEWith Logits Loss' but does not specify version numbers for any software dependencies, libraries, or frameworks used in the experiments. |
| Experiment Setup | Yes | A 6-layer deep 256-unit wide Re LU network f is trained to regress λ... with N = 200 input samples... training progresses with full-batch gradient descent. |