Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Active Learning within Constrained Environments through Imitation of an Expert Questioner
Authors: Kalesha Bullard, Yannick Schroecker, Sonia Chernova
IJCAI 2019 | Venue PDF | LLM Run Details | Input Tokens: 15,493 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 5,358 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments are conducted on a concept learning task to test generalization of the proposed algorithm to different environmental conditions and analyze how time and resource constraints impact efficacy of solving the learning problem. |
| Researcher Affiliation | Academia | Kalesha Bullard , Yannick Schroecker and Sonia Chernova Georgia Institute of Technology EMAIL, EMAIL |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statement or link for the availability of its source code. |
| Open Datasets | Yes | This task was extracted from the University of Washington RGBD dataset of common household objects [Lai et al., 2011]. |
| Dataset Splits | Yes | Each of the training and test task samples are 80 images and 40 images respectively for the prepare-lunch task and 3200 images and 800 images respectively for the pack-lunchbox task. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., GPU models, CPU types, memory specifications). |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | All action types were assigned an a-priori cost, which is 2 for demo queries and feature subset queries, 1 for label queries, and 0 for no query. For training of DT-task-env decision feature weights, an expert questioner was given a very constrained query budget (15) and time allocation (30 turns) to ground the prepare-lunch task concepts. During IRL training, the maximum number of iterations was set to 100, and we selected the set of weights w that performed best on the validation set. |