Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The Shift Toward Open and Reproducible AI Research," Under Review, 2026. doi: 10.48550/arXiv.2606.16974..
Empirical Analysis of Foundational Distinctions in Linked Open Data
Authors: Luigi Asprino, Valerio Basile, Paolo Ciancarini, Valentina Presutti
IJCAI 2018 | Venue PDF | LLM Run Details | Input Tokens: 18,764 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 5,086 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report on a set of experiments based on machine learning and crowdsourcing that show promising results. In this study, we perform a set of experiments to assess whether machines can learn to perform foundational distinctions, and if they match common sense. |
| Researcher Affiliation | Academia | 1 STLab, ISTC-CNR, Rome, Italy 2 University of Bologna, Bologna, Italy 3 University of Turin, Turin, Italy |
| Pseudocode | No | The paper describes the methods in prose and does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The resulting datasets include a sample of annotated DBpedia entities and are available online 17. 17https://github.com/fdistinctions/ijcai18 |
| Open Datasets | Yes | The resulting datasets include a sample of annotated DBpedia entities and are available online 17. 17https://github.com/fdistinctions/ijcai18. We use DBpedia (release 2016-10) in our study as most LOD datasets link to it. |
| Dataset Splits | Yes | We used a 10-fold cross validation strategy using the reference datasets (cf. Section 4). Before training the classi๏ฌers, the datasets were adjusted in order to balance the samples of the two classes. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU models, or cloud instance specifications used for running the experiments. |
| Software Dependencies | No | We have used WEKA16 for their implementation. (Footnote 16: https://www.cs.waikato.ac.nz/ml/weka/) - The paper mentions WEKA but does not specify a version number for it or any other software libraries used. |
| Experiment Setup | No | The paper describes the features used for the machine learning models and the cross-validation strategy, but it does not provide specific hyperparameters (e.g., learning rate, batch size, epochs) or detailed system-level training settings for the algorithms. |