Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The Shift Toward Open and Reproducible AI Research," Under Review, 2026. doi: 10.48550/arXiv.2606.16974..
Efficient Sequential Model-Based Fault-Localization with Partial Diagnoses
Authors: Kostyantyn Shchekotykhin, Thomas Schmitz, Dietmar Jannach
IJCAI 2016 | Venue PDF | LLM Run Details | Input Tokens: 17,312 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,721 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | An experimental evaluation on different benchmark problems shows that our sequential diagnosis approach needs considerably less computation time when compared with an existing domain-independent approach. |
| Researcher Affiliation | Academia | Kostyantyn Shchekotykhin,1 Thomas Schmitz,2 and Dietmar Jannach2 1Alpen-Adria University Klagenfurt, Austria e-mail: EMAIL 2TU Dortmund, Germany e-mail: {๏ฌrstname.lastname}@tu-dortmund.de |
| Pseudocode | Yes | Algorithm 1 summarizes our approach, which in contrast to previous works can operate on the basis of partial diagnoses. |
| Open Source Code | No | The paper states that 'The algorithms were implemented in Java' but does not provide any link, repository, or explicit statement of code availability for the methodology described. |
| Open Datasets | Yes | We evaluated our method on two sets of benchmark problems: (a) the ontologies of the OAEI Conference benchmark as used in [Shchekotykhin et al., 2014], (b) the systems of the DX Competition (DXC) 2011 Synthetic Track. |
| Dataset Splits | No | The paper describes running experiments multiple times with randomly selected preferred diagnoses or scenarios, but it does not specify explicit training, validation, and testing dataset splits. |
| Hardware Specification | No | All tests were performed on a modern laptop computer. This description is too vague and does not provide specific hardware details (e.g., CPU, GPU, memory). |
| Software Dependencies | No | The algorithms were implemented in Java. Choco was used as a constraint solver and Hermi T as Description Logic reasoner. However, specific version numbers for Java, Choco, or Hermi T are not provided. |
| Experiment Setup | Yes | For both strategies, we set the number of diagnoses n that are used to determine the optimal query to 9 as done in [Shchekotykhin et al., 2014], and used the best-performing Entropy strategy for query selection (GETQUERY). |