Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Efficiently Explaining CSPs with Unsatisfiable Subset Optimization
Authors: Emilio Gamba, Bart Bogaerts, Tias Guns
IJCAI 2021 | Venue PDF | LLM Run Details | Input Tokens: 17,808 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,164 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We now experimentally validate the performance of the different versions of our algorithm. Our benchmarks were run on a compute cluster, where each explanation sequence generation was assigned a single core on a 10-core INTEL Xeon Gold 61482 (Skylake) processor, a timelimit of 120 minutes and a memory-limit of 4GB. |
| Researcher Affiliation | Academia | 1Vrije Universiteit Brussel, Belgium 2KU Leuven, Belgium |
| Pseudocode | Yes | Algorithm 1: EXPLAIN-ONE-STEP(C, f, I, Iend) |
| Open Source Code | Yes | Everything was implemented in Python on top of Py SAT1 and is available at https://github. com/ML-KULeuven/ocus-explain. |
| Open Datasets | Yes | All of our experiments were run on a direct translation to Py SAT of the 10 puzzles of Bogaerts et al. [2020]2. |
| Dataset Splits | No | No explicit mention of specific train/validation/test dataset splits, percentages, or counts for reproduction. |
| Hardware Specification | Yes | Our benchmarks were run on a compute cluster, where each explanation sequence generation was assigned a single core on a 10-core INTEL Xeon Gold 61482 (Skylake) processor, a timelimit of 120 minutes and a memory-limit of 4GB. |
| Software Dependencies | Yes | For MIP calls, we used Gurobi 9.0, for SAT calls Mini Sat 2.2 and for Max SAT calls RC2 as bundled with Py SAT (version 0.1.6.dev11). |
| Experiment Setup | Yes | We used a cost of 60 for puzzle-agnostic constraints; 100 for puzzle-specific constraints; and cost 1 for facts. |