Subpopulation selection and debiased estimation for causal inference and predictive model evaluation

Bias in statistical and machine learning models refers to systematic errors that can skew results, leading to inaccurate conclusions and predictions. It can arise from various sources, including data collection methods, model selection, and underlying assumptions. Debiasing techniques aim to mitigate these errors and improve the reliability and validity of models. The goal of debiasing is to enhance model performance by reducing systematic errors, thus providing more accurate and trustworthy results for decision-making and inference. Common strategies for debiasing include cross-validation and bootstrap. Other methods involve correcting for biases introduced by extreme values, imbalanced datasets, and confounding variables. In this thesis, two chapters focus on advancing estimator debias methodologies in causal inference and classification, particularly in the realms of average treatment effect estimation and classification model evaluation. The first chapter introduces a method for interpretable weighted average treatment effect (WATE) estimation under possible violation of overlapping assumption. By selecting subpopula- tions through optimizing a loss function via integer linear programming, the approach enhances the precision and reliability of treatment effect estimates, especially in the presence of extreme propensity scores. Simulation studies show that the proposed WATE estimators outperform classic IPW estimators in terms of bias reduction by a significant magnitude. In the second chapter, classification model evaluation is examined, particularly being focused on cross-validation and bootstrap methods. Previously, our case study demonstrates the enhanced prediction of preterm birth that incorporates two nested cross-validated machine learning classifi- cation models. As a beginning, this chapter delves into the theoretical aspects of cross-validation for linear regression, confirming that sample mean square error is an unbiased consistent estimator of true MSE. After so, this chapter explores the AUC metric for evaluating various classifiers performances and addresses the limitations of the DeLong test, proposing alternatives like bootstrap resampling. Finally, this chapter highlights the issue of bias in nested model AUC difference estimation and demonstrates that sample splitting involvement provides unbiased, asymptotically normal estimators for cross-validated generalized linear model (GLM) and linear discriminant analysis (LDA).

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Statistics

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Liu, Yuxuan

Contributors dc:contributor

• Simpson, Douglas
• Li, Xinran
• Zhu, Ruoqing
• Culpepper, Steven Andrew

• Causal Inference
• Debiased Estimation
• Classification Evaluation

Statement dc:rights

• © 2024 Yuxuan Liu

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/127347

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F127347