Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Retiring Adult: New Datasets for Fair Machine Learning
Authors: Frances Ding, Moritz Hardt, John Miller, Ludwig Schmidt
NeurIPS 2021 | Venue PDF | LLM Run Details | Input Tokens: 17,417 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 5,724 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We highlight a broad initial sweep of new empirical insights relating to trade-offs between fairness criteria, performance of algorithmic interventions, and the role of distribution shift based on our new datasets. |
| Researcher Affiliation | Collaboration | Frances Ding UC Berkeley Moritz Hardt UC Berkeley John Miller UC Berkeley Ludwig Schmidt Toyota Research Institute |
| Pseudocode | No | The paper does not contain any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The datasets and Python package are available for download at https://github.com/zykls/ folktables. |
| Open Datasets | Yes | The datasets and Python package are available for download at https://github.com/zykls/ folktables. |
| Dataset Splits | Yes | The UCI Machine Learning Repository contributed to this development by providing researchers with numerous datasets each with a fixed training and testing split. ... Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please see Appendix C. |
| Hardware Specification | Yes | Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Please see Appendix C. |
| Software Dependencies | No | The paper mentions a 'Python package called folktables' and various machine learning methods, but does not specify key software components with their version numbers required for reproduction. |
| Experiment Setup | Yes | Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please see Appendix C. |