Notice: This website is continuously updated as new conference proceedings and journal volumes are published. Consequently, the figures presented here may differ from those reported in our publications.
A brief overview of our methodology is provided on the Methods page, along with references to publications documenting our approach and links to the code and data used in each study.
The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates.
Lookahead Counterfactual Fairness
Authors: Zhiqun Zuo, Tian Xie, Xuwei Tan, Xueru Zhang, Mohammad Mahdi Khalili
TMLR 2024 | Venue PDF | LLM Run Details | Input Tokens: 28,472 Total number of tokens sent to the LLM as input for this paper's analysis. | Output Tokens: 4,452 Total number of tokens produced by the LLM (including reasoning/thinking tokens) for this paper's analysis.
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on both synthetic and real data to validate the proposed algorithm. We conduct experiments on both synthetic and real data to validate the proposed method. |
| Researcher Affiliation | Academia | Zhiqun Zuo EMAIL Department of Computer Science and Engineering The Ohio State University Tian Xie EMAIL Department of Computer Science and Engineering The Ohio State University Xuwei Tan EMAIL Department of Computer Science and Engineering The Ohio State University Xueru Zhang EMAIL Department of Computer Science and Engineering The Ohio State University Mohammad Mahdi Khalili EMAIL Department of Computer Science and Engineering The Ohio State University |
| Pseudocode | Yes | Algorithm 1 Training a predictor with perfect LCF Input: Training data D = {(x(i), y(i), a(i))}n i=1, response parameter η. 1: Estimate the structural equations 7 using D to determine parameters α, β, w, and γ. 2: For each data point (x(i), y(i), a(i)), draw m samples u(i)[j] m j=1 from conditional distribution Pr{U|X = x(i), A = a(i)} and generate counterfactual ˇy(i)[j] associated with u(i)[j] based on structural equations 7. 3: Compute p1 1 2η(||w α||2 2+γ2). 4: Solve the following optimization problem, ˆp2, ˆp3, ˆθ = arg min p2,p3,θ j=1 l g ˇy(i)[j], u(i)[j] , y(i) where g ˇy(i)[j], u(i)[j] = p1 ˇy(i)[j] 2 + p2ˇy(i)[j] + p3 + hθ(u), θ is a parameter for function h, and l is a loss function. Output: ˆp2, ˆp3, ˆθ |
| Open Source Code | Yes | 1The code for this paper is available in https://github.com/osu-srml/LCF. |
| Open Datasets | Yes | We further measure the performance of our proposed method using the Law School Admission Dataset Wightman (1998). We measure the performance of our proposed method using Loan Prediction Problem Dataset (kag). Loan Prediction Problem Dataset kaggle.com. https://www.kaggle.com/datasets/ altruistdelhite04/loan-prediction-problem-dataset. [Accessed 20-10-2024]. |
| Dataset Splits | Yes | We split the dataset into the training/validation/test set at 60%/20%/20% ratio randomly and repeat the experiment 5 times. We partitioned the data into training, validation, and test sets with 60%/20%/20% ratio. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) are provided for running the experiments. The text mentions training with Adam optimizer and MCMC method, but not the hardware these were run on. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., Python 3.8, PyTorch 1.9) are provided. The paper mentions 'Adam optimization' and 'Markov chain Monte Carlo (MCMC) method Geyer (1992)' but without specific software names or versions. |
| Experiment Setup | Yes | Based on our observation, Adam optimization with a learning rate equal to 10 3 and 2000 epochs gives us the best performance. To train g(ˇy, u), we follows Algorithm 1 with m = 100. For each given data, we sampled m = 500 different k s from this conditional distribution. |