Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Online Learning with Abstention
Authors: Corinna Cortes, Giulia DeSalvo, Claudio Gentile, Mehryar Mohri, Scott Yang
ICML 2018 | Venue PDF | LLM Run Details | Input Tokens: 17,881 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 2,821 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We further report the results of a series of experiments demonstrating that UCB-GT largely outperforms that extension of UCB-N, as well as other standard baselines. |
| Researcher Affiliation | Collaboration | 1Google Research, New York, NY. 2INRIA Lille Nord Europe. 3Courant Institute of Mathematical Sciences, New York, NY. 4D. E. Shaw & Co., New York, NY. |
| Pseudocode | Yes | ALGORITHM 1: UCB-NT... ALGORITHM 2: UCB-GT |
| Open Source Code | No | The paper does not provide any links to open-source code for the methodology described. |
| Open Datasets | Yes | We used the following eight datasets from the UCI data repository: HIGGS, phishing, ijcnn, covtype, eye, skin, cod-rna, and guide. We also used the CIFAR dataset from (Krizhevsky et al., 2009) |
| Dataset Splits | No | The paper does not explicitly provide training/validation/test dataset splits. It mentions 'averaged over five random draws of the data'. |
| Hardware Specification | No | The paper does not provide specific hardware details used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | The predictors h are hyperplanes centered at the origin whose normal vector in Rd is drawn randomly from the Gaussian distribution, N(0, 1)d, where d is the dimension of the feature space of the dataset. The abstention functions r are concentric annuli around the origin with radii in (0, d 20 . . . , d). For each dataset, we generated a total of K = 2,100 experts and all the algorithms were tested for a total of T = 10,000 rounds. We report these results for c 2 {0.1, 0.2, 0.3}. |