Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Out-of-Distribution Generalization by Neural-Symbolic Joint Training
Authors: Anji Liu, Hongming Xu, Guy Van den Broeck, Yitao Liang
AAAI 2023 | Venue PDF | LLM Run Details | Input Tokens: 17,082 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 5,282 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate that our proposed framework is capable of achieving superior outof-distribution generalization performance on two tasks: (i) learning multi-digit addition, where it is trained on short sequences of digits and tested on long sequences of digits; (ii) predicting the optimal action in the Tower of Hanoi, where the model is challenged to discover a policy independent of the number of disks in the puzzle. [...] In this section, we evaluate the proposed NTOC model on the multi-digit addition task and the Tower of Hanoi action prediction task. |
| Researcher Affiliation | Collaboration | Anji Liu2, Hongming Xu3, Guy Van den Broeck2, Yitao Liang1,3 1 Institute for Artificial Intelligence, Peking University 2 Computer Science Department, University of California, Los Angeles 3 Beijing Institute of General Artificial Intelligence (BIGAI) EMAIL, EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1: Symbolic Rule Search |
| Open Source Code | Yes | Our code can be found at https://github.com/sbx126/NToC. |
| Open Datasets | Yes | Given two numbers represented by a sequence of MNIST images, the goal is to predict the sum of both numbers. [...] For all methods, 100 MNIST images are given for pretraining. |
| Dataset Splits | No | The paper mentions training and testing sets ('training the models on numbers of length 3 and test them with numbers of length 5-20', '1,000 training samples, while testing them on 10,000 samples') but does not specify a separate validation split or its size/percentage. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory, or cloud instance types) used for running experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python version, library versions like PyTorch or TensorFlow). |
| Experiment Setup | No | The paper provides minimal experimental setup details such as the number of pretraining samples (e.g., '100 MNIST images are given for pretraining', '500 samples to teach the model') and training samples ('1,000 training samples'), but lacks specific hyperparameters like learning rates, batch sizes, optimizers, or detailed training schedules. |