Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Can We Infer Confidential Properties of Training Data from LLMs?
Authors: Pengrun Huang, Chhavi Yadav, Kamalika Chaudhuri, Ruihan Wu
NeurIPS 2025 | Venue PDF | LLM Run Details | Input Tokens: 25,310 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 2,615 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical evaluations across multiple pretrained LLMs show the success of our attacks, revealing a previously unrecognized vulnerability in LLMs. We release our code at github.com/Pengrun H/Property_inference_attack_LLM. |
| Researcher Affiliation | Academia | Pengrun Huang, Chhavi Yadav, Kamalika Chaudhuri , Ruihan Wu University of California, San Diego EMAIL |
| Pseudocode | No | The paper describes the methods and attacks in prose, but does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks with structured steps. |
| Open Source Code | Yes | We release our code at github.com/Pengrun H/Property_inference_attack_LLM. |
| Open Datasets | Yes | Built on the Chat Doctor dataset, our benchmark includes a range of property types and task configurations. 1https://huggingface.co/datasets/Pengrun/Prop Infer_dataset |
| Dataset Splits | Yes | We use 15,000 samples to train the target models and the remaining 14,791 as auxiliary data for evaluating attacks in the grey-box setting. For medical diagnosis attributes, we use the original training split of the Chat Doctor dataset with size 50, 000 for training the target models |
| Hardware Specification | Yes | All experiments are conducted on NVIDIA RTX 6000 Ada GPU. |
| Software Dependencies | No | The paper mentions software like XGBoost [7] and scikit-learn library [23], but does not provide specific version numbers for these or other key software components. |
| Experiment Setup | Yes | We used the Lo RA [15] method for fine-tuning with a learning rate of 1e 4, dropout rate of 0.05, Lo RA rank of 128 and 5 epochs. |