Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Playgol: Learning Programs Through Play
Authors: Andrew Cropper
IJCAI 2019 | Venue PDF | LLM Run Details | Input Tokens: 15,908 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,992 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally test our claim on two domains: robot planning and real-world string transformations. Our experimental results suggest that playing can substantially improve learning performance. |
| Researcher Affiliation | Academia | Andrew Cropper University of Oxford EMAIL |
| Pseudocode | Yes | Algorithm 1 shows the Playgol algorithm |
| Open Source Code | No | The paper mentions that Playgol is implemented and uses Metagol (with a link to Metagol's GitHub), but it does not provide an explicit statement or link for the Playgol source code itself. |
| Open Datasets | Yes | Our dataset is based on the dataset from [Lin et al., 2014], which in turn is based on [Gulwani, 2011]. |
| Dataset Splits | No | The paper specifies 5 training examples and 5 testing examples per task in the string transformation experiment, but does not mention a separate validation split or explicit percentages for the splits. |
| Hardware Specification | No | The paper mentions a timeout of 60 seconds per task but does not provide any specific details about the hardware (e.g., CPU, GPU models, memory) used for the experiments. |
| Software Dependencies | No | The paper states 'Playgol uses Metagol [Cropper and Muggleton, 2016b]', but it does not specify a version number for Metagol or any other software dependencies. |
| Experiment Setup | Yes | For each p in {0, 200, 400, . . . , 2000}, we call playgol(Xn,BK,Tb,p,5) which returns a set of programs Pp. We enforce a timeout of 60 seconds per play and build task. |