Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The Shift Toward Open and Reproducible AI Research," Under Review, 2026. doi: 10.48550/arXiv.2606.16974..
Adaptive Data Analysis with Correlated Observations
Authors: Aryeh Kontorovich, Menachem Sadigurschi, Uri Stemmer
ICML 2022 | Venue PDF | LLM Run Details | Input Tokens: 24,589 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,073 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We embark on a systematic study of the possibilities of adaptive data analysis with correlated observations. First, we show that, in some cases, differential privacy guarantees generalization even when there are dependencies within the sample, which we quantify using a notion we call Gibbs-dependence. We complement this result with a tight negative example. Second, we show that the connection between transcript-compression and adaptive data analysis can be extended to the non-iid setting. |
| Researcher Affiliation | Collaboration | Aryeh Kontorovich 1 Menachem Sadigurschi 1 Uri Stemmer 2 3 1Department of Computer Science, Ben Gurion University 2Blavatnik School of Computer Science, Tel Aviv University 3Google Research. |
| Pseudocode | Yes | Algorithm 1 Game(M, k, A, S) and Algorithm 2 Deviating Private Algorithm |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code or links to a code repository. |
| Open Datasets | No | The paper defines theoretical concepts based on 'a sample S µ containing n (possibly correlated) observations' or 'a distribution µ over n-tuples', but does not mention the use of specific, publicly available datasets for empirical training or evaluation. |
| Dataset Splits | No | The paper is theoretical and does not describe empirical experiments with dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper is theoretical and does not provide any details about hardware specifications used for experiments. |
| Software Dependencies | No | The paper focuses on theoretical analysis and does not specify software dependencies with version numbers for reproducibility. |
| Experiment Setup | No | The paper is theoretical and does not describe an experimental setup including hyperparameters or specific training configurations. |