Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
CLIP-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experiments
Authors: Jessica Dai, Paula Gradu, Christopher Harshaw
NeurIPS 2023 | Venue PDF | LLM Run Details | Input Tokens: 33,577 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 3,954 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To complement our theoretical results, we conduct simulations using data from a microeconomic experiment. In Section 7, we support these theoretical results with simulations using data from a microeconomic experiment. |
| Researcher Affiliation | Academia | Jessica Dai UC Berkeley EMAIL Paula Gradu UC Berkeley EMAIL Christopher Harshaw MIT EMAIL |
| Pseudocode | Yes | Algorithm 1: CLIP-OGD |
| Open Source Code | Yes | A repository for reproducing simulations is: https://github.com/crharshaw/Clip-OGD-sims |
| Open Datasets | Yes | We evaluate the performance of CLIP-OGD and Explore-then-Commit (ETC) for the purpose of Adaptive Neyman Allocation on the field experiment of Groh and Mc Kenzie [2016], which investigates the effect of macro-insurance on micro-enterprises in post-revolution Egypt. |
| Dataset Splits | No | The paper does not explicitly provide details about train, validation, or test dataset splits. It mentions using "the first T units in the sequence" as the population for a given T, and that "Units are shuffled to appear in an arbitrary order and outcomes are normalized." |
| Hardware Specification | Yes | Simulations were run on a 2019 Mac Book Pro with 2.4 GHz Quad-Core Intel Core i5 and 16 GB LPDDR3 RAM. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python version, library versions) used for the experiments. |
| Experiment Setup | Yes | CLIP-OGD is run with the parameters recommended in Theorem 4.2 and ETC is run with T0 T 1{3 so that the exploration phase grows with T. Theorem 4.2 states parameter values η a 1{T and α a 5 logp Tq. |