Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Optimal and Fair Encouragement Policy Evaluation and Learning
Authors: Angela Zhou
NeurIPS 2023 | Venue PDF | LLM Run Details | Input Tokens: 29,511 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 5,409 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 6 Experiments", "Our case study is on a dataset of judicial decisions on supervised release based on risk-score-informed recommendations [38].", "In Figure 1 we provide descriptive information illustrating heterogeneity (including by protected attribute) in adherence and effectiveness.", "In Figure 2 we highlight results from constrained policy optimization. |
| Researcher Affiliation | Academia | Angela Zhou Department of Data Sciences and Operations University of Southern California EMAIL |
| Pseudocode | Yes | Algorithm 1 MW2REDFAIR(D, g, E, M, d)", "Algorithm 2 Two-stage localized fair classification via reductions |
| Open Source Code | No | The paper does not contain an explicit statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | Our case study is on a dataset of judicial decisions on supervised release based on risk-score-informed recommendations [38].", "The Oregon Health Insurance Study [16] is an important study on the causal effect of expanding public health insurance on healthcare utilization, outcomes, and other outcomes. |
| Dataset Splits | Yes | Algorithm 2 Two-stage localized fair classification via reductions 1: Randomly split the data into D1, D2", "We split the data, learn nuisance estimators η1 for use in our policy value and constraint estimates, and run Algorithm 1 (MW2REDFAIR(D1, h, E, M, d; η1)) to obtain an estimate of the optimal policy ˆπ1, and the constraint variances at ˆπ1. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper mentions the use of 'logistic regression' and 'gradient-boosted regression' but does not specify any software or library names with version numbers that would be required for reproducibility. |
| Experiment Setup | No | The paper mentions the types of models used (logistic regression, gradient-boosted regression) and arbitrarily set cost values for the objective function but does not provide specific training hyperparameters such as learning rates, batch sizes, or optimizer settings for these models. |