Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Gradient-Based Optimization for Bayesian Preference Elicitation
Authors: Ivan Vendrov, Tyler Lu, Qingqing Huang, Craig Boutilier10292-10301
AAAI 2020 | Venue PDF | LLM Run Details | Input Tokens: 19,015 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,370 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that our gradientbased EVOI technique achieves state-of-the-art performance across several domains while scaling to large item spaces. |
| Researcher Affiliation | Industry | Ivan Vendrov, Tyler Lu, Qingqing Huang, Craig Boutilier Google Research, Mountain View, California EMAIL |
| Pseudocode | Yes | Algorithm 1 Deep Retr Uniq. Inputs: optimized X and U |
| Open Source Code | No | The paper mentions using "Tensor Flow, Py Torch" as computational frameworks but does not provide any statement about releasing their own source code or a link to a repository. |
| Open Datasets | Yes | Using the Movie Lens 100-K dataset, we train user and movie embeddings with dimension d = 10. ... We use the Movie Lens-20M (Harper and Konstan 2015) dataset and represent each movie with 100 binary attributes from the Tag Genome (Vig, Sen, and Riedl 2012). |
| Dataset Splits | No | The paper uses datasets like Movie Lens 100-K and Goodreads, and describes running elicitation trials with random selections of items and user embeddings, but it does not provide specific train/validation/test dataset split percentages or counts. |
| Hardware Specification | Yes | We benchmark the algorithmic runtimes on a workstation with a 12-core Intel Xeon E5-1650 CPU at 3.6GHz, and 64GB of RAM. |
| Software Dependencies | No | The paper mentions using "Tensor Flow, Py Torch" as computational frameworks but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | We initialize query embeddings to random uniform values in [0, 1]100, then run gradient ascent on Eq. 7 for 100 steps, initializing the regularization weight λ at 0.01 and multiplying λ by 1.1 each iteration. |