Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Online Learning to Rank with Top-k Feedback
Authors: Sougata Chaudhuri, Ambuj Tewari
JMLR 2017 | Venue PDF | LLM Run Details | Input Tokens: 41,226 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 3,473 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically demonstrate the performance of our algorithms on simulated and real world data sets. |
| Researcher Affiliation | Academia | Department of Statistics 1 Department of Electrical Engineering and Computer Science 2 University of Michigan Ann Arbor, MI 48109, USA |
| Pseudocode | Yes | Algorithm 1 Rankingwith Top-k Feedback(RTop-k F)Non Contextual |
| Open Source Code | No | The paper does not contain any explicit statement or link indicating the release of source code for the methodology described. |
| Open Datasets | Yes | We compared the various ranking functions on two large scale commercial data sets. They were Yahoo s Learning to Rank Challenge data set (Chapelle and Chang, 2011) and a data set published by Russian search engine Yandex (IM-2009). The Yahoo data set had 19944 unique queries with 5 distinct relevance levels, while Yandex had 9126 unique queries with 5 distinct relevance levels. |
| Dataset Splits | No | The paper describes an online learning setting, where data is processed sequentially over time (T rounds or iterations), rather than partitioned into explicit, fixed training, validation, and test sets. Therefore, traditional dataset splits are not provided in the context of this online learning problem. |
| Hardware Specification | No | No specific hardware details such as CPU, GPU models, or memory specifications are mentioned in the paper. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers, such as programming languages, libraries, or solvers. |
| Experiment Setup | Yes | All the online algorithms, other than the fully random one, involve learning rate η and exploration parameter γ... In our experiments, for each instance of Algorithm 2, we selected a time varying η = 0.01 t2/3 and γ = 0.1 t1/3 , for round t . We fixed ϵ = 0.01. For List Net, we selected η = 0.01 t1/2 , since regret guarantee in OGD is established with η = O(T 1/2). |