Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
A Kernel Stein Test of Goodness of Fit for Sequential Models
Authors: Jerome Baum, Heishiro Kanagawa, Arthur Gretton
ICML 2023 | Venue PDF | LLM Run Details | Input Tokens: 25,757 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 3,777 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our test is shown to perform well in practice on discrete sequential data benchmarks. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University College London, UK 2Gatsby Computational Neuroscience Unit, UCL 3School of Mathematics, Statistics and Physics, Newcastle University, UK. |
| Pseudocode | Yes | Algorithm 1 Parametric bootstrap test ... Algorithm 2 Wild bootstrap test |
| Open Source Code | Yes | The code is available at https://github.com/test-for-sequential-models/code |
| Open Datasets | No | The paper describes generating synthetic data for its experiments based on various models (e.g., Markov chains, random walks), but it does not specify public datasets or provide access information (links, DOIs, or citations) for pre-existing public datasets used for training. |
| Dataset Splits | No | The paper performs goodness-of-fit tests on generated samples. It does not mention train/validation/test splits or specific percentages/counts for data partitioning in the context of model training and evaluation. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments, such as GPU/CPU models, memory, or cloud computing instances. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | For the KSD test we use the J-location modification neighborhood (6) with J = ; i.e., we allow edits anywhere in the sequence. ... We choose S = {0, . . . , m 1} for our alphabet, with m = 8. ... The chain terminates with probability 1/20 after each step. ... For the KSD, we use the ZS and ZS configurations as in Section 4.2 with J = 1. We choose sparse, point-dependent insertion and substitution neighborhoods; we use 16 high-probability tokens for a given context. |