Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Statistical Inference and A/B Testing for First-Price Pacing Equilibria
Authors: Luofeng Liao, Christian Kroer
ICML 2023 | Venue PDF | LLM Run Details | Input Tokens: 44,365 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 3,090 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical simulations verify our central limit theorems, and empirical coverage rates for our confidence intervals agree with our theory. |
| Researcher Affiliation | Academia | 1IEOR, Columbia University. Correspondence to: Luofeng Liao <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 A/B test effect of a new feature on revenue |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | No | The paper describes generating synthetic data for simulations ( |
| Dataset Splits | No | The paper describes statistical simulations to verify theorems, not traditional machine learning training/validation/test splits of a pre-existing dataset. No specific dataset splits are mentioned for validation purposes. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used to run the experiments. |
| Software Dependencies | No | The paper does not list any specific software dependencies with version numbers. |
| Experiment Setup | Yes | We clearly see that (i) if β i < 1 then the finite sample distribution is close to a normal distribution, and (ii) if β i = 1 (or very close to 1, such as β14,21 in the uniform value plots, β20,23 in exponential), the finite sample distribution puts most of the probability mass at 1. For cases where β i is close to 1, we need to futher increase number of items to observe normality. |