Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Decoding As Dynamic Programming For Recurrent Autoregressive Models
Authors: Najam Zaidi, Trevor Cohn, Gholamreza Haffari
ICLR 2020 | Venue PDF | LLM Run Details | Input Tokens: 15,617 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,554 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on the text in๏ฌlling task over SWAG and Daily Dialogue datasets show that our decoding method is superior to strong competing decoding methods. |
| Researcher Affiliation | Academia | Najam Zaidi Faculty of Information Technology Monash University, Australia EMAIL Trevor Cohn School of Computing and Information Systems University of Melbourne, Australia EMAIL Gholamreza Haffari Faculty of Information Technology Monash University, Australia EMAIL |
| Pseudocode | No | The paper describes algorithmic steps in narrative form but does not present structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our methods and the baslines are implemented on top of Open NMT1 (Klein et al., 2017). 1https://github.com/Najamxaidi/Decoding-as-a-dynamic-program-for-recurrent-autoregressive-models.git |
| Open Datasets | Yes | We conduct experiments on two datasets: SWAG (Zellers et al., 2018) and Daily dialogue (Li et al., 2017) |
| Dataset Splits | No | The paper mentions training and testing but does not explicitly provide details about validation dataset splits (e.g., percentages or counts). |
| Hardware Specification | No | The paper mentions using 'computational infrastructure' from MASSIVE but does not provide specific details on the hardware used, such as GPU/CPU models, memory, or the number of machines. |
| Software Dependencies | No | The paper states that methods are 'implemented on top of Open NMT' but does not provide specific version numbers for Open NMT or other software dependencies. |
| Experiment Setup | Yes | All models are trained with a word embedding size and hidden dimension size of 512. We use ADAM optimiser to train the models with an initial learning rate of 0.001. ... The models were trained for 10 epochs. We use the Nesterov optimiser with a learning rate of 0.1. ... All ยต s for the penalty terms corresponding to different constraints, are initialised with 0.5 and are multiplied by 1.2 after 5 iterations, and decoding was run for 10 iterations. |