Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Near-Optimal Goal-Oriented Reinforcement Learning in Non-Stationary Environments
Authors: Liyu Chen, Haipeng Luo
NeurIPS 2022 | Venue PDF | LLM Run Details | Input Tokens: 18,764 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 3,209 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We initiate the study of dynamic regret minimization for goal-oriented reinforcement learning modeled by a non-stationary stochastic shortest path problem with changing cost and transition functions. We start by establishing a lower bound Ω((B SAT ( c + B2 P ))1/3K2/3)...These algorithms combine the ideas of finite-horizon approximation [Chen et al., 2022a], special Bernstein-style bonuses of the MVP algorithm [Zhang et al., 2020], adaptive confidence widening [Wei and Luo, 2021], as well as some new techniques such as properly penalizing long-horizon policies. |
| Researcher Affiliation | Academia | Liyu Chen University of Southern California EMAIL Haipeng Luo University of Southern California EMAIL |
| Pseudocode | Yes | Algorithm 1 Finite-Horizon Approximation of SSP Algorithm 2 Non-Stationary MVP Algorithm 3 Non-Stationary MVP with a Doubling Trick Algorithm 4 MVP with Non-Stationarity Tests Algorithm 5 A Two-Phase Variant of Algorithm 1 Algorithm 6 MASTER-Base Algorithm (Phase 1) |
| Open Source Code | No | The paper does not provide concrete access to source code. It is a theoretical paper and does not mention code release, links to repositories, or code in supplementary materials. |
| Open Datasets | No | The paper is theoretical and focuses on algorithm design and regret bounds. It does not mention using or training on any specific dataset. The 'Preliminaries' section defines the model but does not refer to empirical datasets. |
| Dataset Splits | No | The paper is theoretical and does not describe experimental validation or data splits (training/test/validation). |
| Hardware Specification | No | The paper is theoretical and does not describe running experiments. Therefore, no hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and focuses on algorithmic design and analysis. It does not mention specific software dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and does not include details about an experimental setup, such as hyperparameters, training configurations, or system-level settings. |