Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The Shift Toward Open and Reproducible AI Research," Under Review, 2026. doi: 10.48550/arXiv.2606.16974..
Fine-Grained Semantic Conceptualization of FrameNet
Authors: Jin-woo Park, Seung-won Hwang, Haixun Wang
AAAI 2016 | Venue PDF | LLM Run Details | Input Tokens: 15,604 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,578 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our extensive analysis with real-life data validates that our approach improves not only the quality of the identified concepts for Frame Net, but also that of applications such as selectional preference. |
| Researcher Affiliation | Collaboration | Jin-woo Park POSTECH, Korea, Republic of EMAIL Seung-won Hwang Yonsei University, Korea, Republic of EMAIL Haixun Wang Facebook Inc., USA EMAIL |
| Pseudocode | Yes | Algorithm 1: Truss (G ) |
| Open Source Code | No | The paper states: "The entire results are released at http://karok.postech.ac.kr/FEconceptualization.zip." This link is for results, not source code. No other explicit statement or link for the paper's source code is provided. |
| Open Datasets | Yes | Proposed method To overcome concept and instance sparsity of manually-built KB, we utilize on Probase (Wu et al. 2012)2, which contains millions of fine-grained concepts automatically harvested from billions of web documents. Footnote 2: Dataset publicly available at http://probase.msra.cn/dataset.aspx. Frame Net (Fillmore, Johnson, and Petruck 2003)3. Footnote 3: We used the Frame Net 1.5 dataset. |
| Dataset Splits | No | The paper describes the Sem Eval 2010 dataset used for pseudo-disambiguation evaluation but does not specify training, validation, or test splits for its model's development and evaluation (e.g., percentages, sample counts, or references to standard splits). |
| Hardware Specification | Yes | All experiments were carried out on a machine with a Intel Core i3 CPU processor at 3.07GHz and 4GB of DDR3 memory. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., library names like PyTorch or TensorFlow with their respective versions). |
| Experiment Setup | No | The paper does not provide specific experimental setup details such as hyperparameters (e.g., learning rate, batch size, number of epochs) or specific training configurations. |