Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Optimal Arms Identification with Knapsacks
Authors: Shaoang Li, Lan Zhang, Yingqi Yu, Xiangyang Li
ICML 2023 | Venue PDF | LLM Run Details | Input Tokens: 27,068 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 3,186 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 6. Numerical Evaluations We consider a specific instance in which there are four arms (m = 4), three types of resources (d = 3), the expected per-round resource constraint of 0 < b 1, and a parameter 0 < ϵ b. The unknown reward vector is r = (0.5, 0.5 ϵ, 0.5, 0.5), and the unknown expected resource consumption is represented by the matrix: Figure 1. The results obtained in different environments. ... The results (accuracy) obtained in different environments are summarized in Figure 1. |
| Researcher Affiliation | Academia | 1University of Science and Technology of China, Hefei, China. |
| Pseudocode | Yes | Algorithm 1 Base OAK Algorithm (BASEOAK) ... Algorithm 2 Full OAK Algorithm (FULLOAK) ... Algorithm 3 BASEOAK |
| Open Source Code | Yes | The code for algorithms could be available at https://github.com/Shaoang Li/ OAK-problem.git. |
| Open Datasets | No | The paper describes a simulated environment/instance rather than using a named, publicly available dataset with concrete access information. 'We consider a specific instance in which there are four arms (m = 4), three types of resources (d = 3), the expected per-round resource constraint of 0 < b 1, and a parameter 0 < ϵ b.' |
| Dataset Splits | No | The paper describes a simulation setup and varying parameters (T, b, ϵ) but does not mention specific training, validation, or test dataset splits or cross-validation for a defined dataset. |
| Hardware Specification | No | The paper describes the setup for numerical evaluations but does not mention any specific hardware components (e.g., CPU, GPU models, memory, or cloud instance types) used for running the simulations. |
| Software Dependencies | No | The paper provides a link to GitHub, implying code for the algorithms. However, it does not explicitly list any specific software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x). |
| Experiment Setup | Yes | We consider a specific instance in which there are four arms (m = 4), three types of resources (d = 3), the expected per-round resource constraint of 0 < b 1, and a parameter 0 < ϵ b. The unknown reward vector is r = (0.5, 0.5 ϵ, 0.5, 0.5), and the unknown expected resource consumption is represented by the matrix: ... We begin by considering the case where the knapsack b = 0.2 and the gap ϵ = 0.01. ... we vary the value of ϵ while keeping b = 0.2 and T = 2 * 10^4 fixed... We conduct experiments with ϵ = 0.01 and T = 2 * 10^4 fixed... |