Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Modeling Source Syntax and Semantics for Neural AMR Parsing
Authors: DongLai Ge, Junhui Li, Muhua Zhu, Shoushan Li
IJCAI 2019 | Venue PDF | LLM Run Details | Input Tokens: 16,009 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,589 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on an English benchmark dataset show that our two approaches achieve significant improvement of 3.1% and 3.4% F1 scores over a strong seq2seq baseline. |
| Researcher Affiliation | Collaboration | Donglai Ge1 , Junhui Li1 , Muhua Zhu2 and Shoushan Li1 1School of Computer Science and Technology, Soochow University, Suzhou, China 2Alibaba Group, Hangzhou, China |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper refers to 'https://github.com/Rik VN/AMR' for pre-processing and post-processing scripts provided by [van Noord and Bos, 2017], but does not state that the source code for their own proposed methodology is open-source or available. |
| Open Datasets | Yes | For evaluation of our approach, we use the sentences annotated with AMRs from the LDC release LDC2017T10. |
| Dataset Splits | Yes | The dataset consists of 36,521 training AMRs, 1,368 development AMRs and 1,371 testing AMRs. |
| Hardware Specification | Yes | In all experiments, we train the models for 250K steps on a single K40 GPU. |
| Software Dependencies | No | The paper mentions 'Allen NLP' and 'tensor2tensor' as tools used, but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | In parameter setting, we set the number of layers in both the encoder and decoder to 6. For optimization we use Adam with β1 = 0.1 [Kingma and Ba, 2015]. The number of heads is set to 8. In addition, we set the hidden size to 512 and the batch token-size to 4096. In beam searching, we increase the extra length as 100 from default 50. We also set Google NMT length penalty parameter α = 1.0 to encourage longer generation. In all experiments, we train the models for 250K steps on a single K40 GPU. |