Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The Shift Toward Open and Reproducible AI Research," Under Review, 2026. doi: 10.48550/arXiv.2606.16974..
Entropy Regularization for Population Estimation
Authors: Ben Chugg, Peter Henderson, Jacob Goldin, Daniel E. Ho
AAAI 2023 | Venue PDF | LLM Run Details | Input Tokens: 14,734 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 5,216 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We run experiments on four publicly available datasets: The Current Population Survey (CPS), the American Community Survey (ACS), a voter turnout dataset, and data on All State severity claims. These four were chosen because they each correspond to a real-world optimize-and-estimate setting. |
| Researcher Affiliation | Academia | 1Carnegie Mellon University 2 Stanford University 3 University of Chicago |
| Pseudocode | Yes | Algorithm 1: Entropy-regularized Pareto Sampling |
| Open Source Code | Yes | Experimental results, datasets, and code can be found at https://github.com/bchugg/ent-reg-pop-est. |
| Open Datasets | Yes | We run experiments on four publicly available datasets: The Current Population Survey (CPS), the American Community Survey (ACS), a voter turnout dataset, and data on All State severity claims. ... More detail on each dataset and further justification for their selection can be found in Appendix B. |
| Dataset Splits | Yes | observations for the first period are selected uniformly at random to provide a initial training set for the model. ... We perform a randomized grid search on a small holdout set to determine a suitable set of hyperparameters for each dataset (see Appendix I for more details). |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., GPU models, CPU types, or memory specifications). |
| Software Dependencies | No | The paper mentions using "random forest regressors" but does not specify version numbers for this or any other software components, libraries, or programming languages used. |
| Experiment Setup | Yes | Experimental Protocol: For each dataset and method, observations for the first period are selected uniformly at random to provide a initial training set for the model. ... We perform a randomized grid search on a small holdout set to determine a suitable set of hyperparameters for each dataset (see Appendix I for more details). ... Throughout our experiments, we keep the budget between approximately 5-10% of the dataset size in each period, i.e., Kt [0.05, 0.1]Xt (depending on the dataset). |