Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Learning Continuous Semantic Representations of Symbolic Expressions
Authors: Miltiadis Allamanis, Pankajan Chanthirasegaran, Pushmeet Kohli, Charles Sutton
ICML 2017 | Venue PDF | LLM Run Details | Input Tokens: 16,658 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,720 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform an exhaustive evaluation on the task of checking equivalence on a highly diverse class of symbolic algebraic and boolean expression types, showing that our model significantly outperforms existing architectures. Experimental evaluation on a highly diverse class of symbolic algebraic and boolean expression types shows that EQNETs dramatically outperform existing architectures like TREENNs and RNNs. We split the datasets into training, validation and test sets. |
| Researcher Affiliation | Collaboration | 1Microsoft Research, Cambridge, UK 2University of Edinburgh, UK 3Deep Mind, London, UK 4The Alan Turing Institute, London, UK. |
| Pseudocode | Yes | Figure 1 (b) COMBINE of EQNET and (c) Loss function used for subexpression autoencoder present structured pseudocode. Additionally, the TREENN function is described in a pseudocode-like format in Section 1: "TREENN (current node n) if n is not a leaf then rn COMBINE(TREENN(c0), . . . , TREENN(ck), τn), where (c0, . . . , ck) = ch(n) else rn LOOKUPLEAFEMBEDDING(τn) return rn" |
| Open Source Code | Yes | Code and data are available at groups.inf.ed.ac.uk/cup/semvec. |
| Open Datasets | Yes | We provide the datasets online at groups.inf.ed.ac.uk/cup/semvec. |
| Dataset Splits | Yes | We split the datasets into training, validation and test sets. ...randomly split the expressions in each class into training, validation, SEENEQCLASS test in the proportions 60% 15% 25%. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. It only mentions "additional computational resources" in the acknowledgements. |
| Software Dependencies | No | The paper does not provide specific version numbers for software components, libraries, or programming languages used in the experiments. |
| Experiment Setup | Yes | Hyperparameters. We tune the hyperparameters of all models using Bayesian optimization... The selected hyperparameters are detailed in the supplementary material. To train the model, we use a max-margin objective that maximizes classification accuracy, i.e. L(T, ei) = LACC(T, ei) + µn Q SUBEXPAE(ch(n), n) ... for each epoch t, we set µ = 1 10 νt with ν 0. |