Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Launch and Iterate: Reducing Prediction Churn
Authors: Mahdi Milani Fard, Quentin Cormier, Kevin Canini, Maya Gupta
NeurIPS 2016 | Venue PDF | LLM Run Details | Input Tokens: 15,032 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 3,744 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on benchmark datasets for different classification algorithms demonstrate the method and the resulting reduction in churn. |
| Researcher Affiliation | Collaboration | Q. Cormier ENS Lyon 15 parvis René Descartes Lyon, France EMAIL M. Milani Fard, K. Canini, M. R. Gupta Google Inc. 1600 Amphitheatre Parkway Mountain View, CA 94043 EMAIL |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any information about open-source code being made available for the described methodology. |
| Open Datasets | Yes | This section demonstrates the churn reduction effect of the RCP operator for three UCI benchmark datasets (see Table 2) with three regression algorithms: ridge regression, random forest regression, and support vector machine regression with RBF kernel, all implemented in Scikit-Learn [12]. [...] Nomao [13] News Popularity [14] Twitter Buzz [15] |
| Dataset Splits | Yes | We randomly split each dataset into three fixed parts: a training set, a validation set on which we optimized the hyper-parameters for all algorithms, and a testing set. [...] Table 2: Validation set 1000 samples (for each dataset) |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU, GPU models, or memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions 'all implemented in Scikit-Learn [12]' but does not specify a version number for Scikit-Learn or any other software dependencies. |
| Experiment Setup | Yes | We used fixed values of α = 0.5 and ϵ = 0.5 for all the experiments in Table 3 [...] The dataset perturbation sub-samples 80% of the examples in TA and randomly drops 3-7 features. We run 40 independent chains to measure the variability, and report the average outcome and standard deviation. [...] See the supplementary material for more experimental details. |