Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
How Private are DP-SGD Implementations?
Authors: Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Amer Sinha, Chiyuan Zhang
ICML 2024 | Venue PDF | LLM Run Details | Input Tokens: 21,681 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 5,870 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | For example, in Figure 1, we fix δ = 10−6 and the number of steps T = 10,000,3 and compare the value of εB(δ) for various values of σ. For σ = 0.5, we find εP(δ) < 1.96 (PLD) and εP(δ) < 3.43 (RDP), but εS(δ) > 10.994 and εD(δ) ≈ 10.997. We provide the IPython notebook4 that was used for all the numerical demonstrations; the notebook can be executed using a free CPU runtime on Google Colab. |
| Researcher Affiliation | Industry | 1Google Research. Correspondence to: Pritish Kamath <EMAIL>, Pasin Manurangsi <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 ABLQB: Adaptive Batch Linear Queries |
| Open Source Code | Yes | We provide the IPython notebook4 that was used for all the numerical demonstrations; the notebook can be executed using a free CPU runtime on Google Colab.4https://colab.research.google.com/drive/1z_I2H8YEXb_Qy_D6g_ZVVsk_FwcOiM5YMvqRe?usp=sharing |
| Open Datasets | No | The paper is a theoretical/analytical work that does not utilize named, publicly available datasets for training or evaluation. It defines abstract 'datasets' in the context of differential privacy mechanisms (e.g., 'Input: Dataset x = (x1, . . . , xn)'). |
| Dataset Splits | No | The paper does not involve training models on datasets that require explicit train/validation/test splits. Its experiments are numerical computations based on varying parameters. |
| Hardware Specification | No | The paper states that its numerical demonstrations can be executed using a 'free CPU runtime on Google Colab' but does not provide specific hardware details such as CPU model, memory, or GPU specifications. |
| Software Dependencies | No | The paper mentions using 'the open source dp accounting library (Google’s DP Library., 2020)' and refers to other frameworks like 'Tensorflow Privacy' and 'Pytorch Opacus'. However, it does not provide specific version numbers for these software dependencies (e.g., 'Python 3.8, PyTorch 1.9'). |
| Experiment Setup | Yes | For example, in Figure 1, we fix δ = 10−6 and the number of steps T = 10,000,3 and compare the value of εB(δ) for various values of σ. For σ = 0.5, we find εP(δ) < 1.96 (PLD) and εP(δ) < 3.43 (RDP), but εS(δ) > 10.994 and εD(δ) ≈ 10.997. |