We propose a novel multi-task neural network approach for estimating distributional treatment effects (DTE) in randomized experiments. While DTE provides more granular insights into the experiment outcomes over conventional methods focusing on the Average Treatment Effect (ATE), estimating it with regression adjustment methods presents significant challenges. Specifically, precision in the distribution tails suffers due to data imbalance, and computational inefficiencies arise from the need to solve numerous regression problems, particularly in large-scale datasets commonly encountered in industry. To address these limitations, our method leverages multi-task neural networks to estimate conditional outcome distributions while incorporating monotonic shape constraints and multi-threshold label learning to enhance accuracy. To demonstrate the practical effectiveness of our proposed method, we apply our method to both simulated and real-world datasets, including a randomized field experiment aimed at reducing water consumption in the US and a large-scale A/B test from a leading streaming platform in Japan. The experimental results consistently demonstrate superior performance across various datasets, establishing our method as a robust and practical solution for modern causal inference applications requiring a detailed understanding of treatment effect heterogeneity.
Macro variables frequently display time-varying distributions, driven by the dynamic and evolving characteristics of economic, social, and environmental factors that consistently reshape the fundamental patterns and relationships governing these variables. To better understand the distributional dynamics beyond the central tendency, this paper introduces a novel semi-parametric approach for constructing time-varying conditional distributions, relying on the recent advances in distributional regression. We present an efficient precision-based Markov Chain Monte Carlo algorithm that simultaneously estimates all model parameters while explicitly enforcing the monotonicity condition on the conditional distribution function. Our model is applied to construct the forecasting distribution of inflation for the U.S., conditional on a set of macroeconomic and financial indicators. The risks of future inflation deviating excessively high or low from the desired range are carefully evaluated. Moreover, we provide a thorough discussion about the interplay between inflation and unemployment rates during the Global Financial Crisis, COVID, and the third quarter of 2023.
Vector autoregression is an essential tool in empirical macroeconomics and finance for understanding the dynamic interdependencies among multivariate time series. In this study, we expand the scope of vector autoregression by incorporating a multivariate distributional regression framework and introducing a distributional impulse response function, providing a comprehensive view of dynamic heterogeneity. We propose a straightforward yet flexible estimation method and establish its asymptotic properties under weak dependence assumptions. Our empirical analysis examines the conditional joint distribution of GDP growth and financial conditions in the United States, with a focus on the global financial crisis. Our results show that tight financial conditions lead to a multimodal conditional joint distribution of GDP growth and financial conditions, and easing financial conditions significantly impacts long-term GDP growth, while improving the GDP growth during the global financial crisis has limited effects on financial conditions.
In this paper, we address the issue of estimating and inferring distributional treatment effects in randomized experiments. The distributional treatment effect provides a more comprehensive understanding of treatment heterogeneity compared to average treatment effects. We propose a regression adjustment method that utilizes distributional regression and pre-treatment information, establishing theoretical efficiency gains without imposing restrictive distributional assumptions. We develop a practical inferential framework and demonstrate its advantages through extensive simulations. Analyzing water conservation policies, our method reveals that behavioral nudges systematically shift consumption from high to moderate levels. Examining health insurance coverage, we show the treatment reduces the probability of zero doctor visits by 6.6 percentage points while increasing the likelihood of 3-6 visits. In both applications, our regression adjustment method substantially improves precision and identifies treatment effects that were statistically insignificant under conventional approaches.
Semiparametric Single-Index Estimation for Average Treatment Effects
Econometric Reviews, 2025 44(6), pp. 843-885, with Difang Huang and Jiti Gao
We propose a semiparametric method to estimate the average treatment effect under the assumption of unconfoundedness given observational data. Our estimation method alleviates misspecification issues of the propensity score function by estimating the single-index link function involved through Hermite polynomials. Our approach is computationally tractable and allows for moderately large dimension covariates. We provide the large sample properties of the estimator and show its validity. Also, the average treatment effect estimator achieves the parametric rate and asymptotic normality. Our extensive Monte Carlo study shows that the proposed estimator is valid in finite samples. Applying our method to maternal smoking and infant health, we find that conventional estimates of smoking's impact on birth weight may be biased due to propensity score misspecification, and our analysis of job training programs reveals earnings effects that are more precisely estimated than in prior work. These applications demonstrate how addressing model misspecification can substantively affect our understanding of key policy-relevant treatment effects.
Keywords: Average treatment effects, Causal inference, Hermite series expansion, Propensity score
We propose a quantile random-coefficient regression with interactive fixed effects to study the effects of group-level policies that are heterogeneous across individuals. Our approach is the first to use a latent factor structure to handle the unobservable heterogeneities in the random coefficient. The asymptotic properties and an inferential method for the policy estimators are established. The model is applied to evaluate the effect of the minimum wage policy on earnings between 1967 and 1980 in the United States. Our results suggest that the minimum wage policy has significant and persistent positive effects on black workers and female workers up to the median. Our results also indicate that the policy helps reduce income disparity up to the median between two groups: black, female workers versus white, male workers. However, the policy is shown to have little effect on narrowing the income gap between low- and high-income workers within the subpopulations.
Keywords: Heterogenous policy effects, Hierarchical regression, Random coefficient model
Not Eliminate but Aggregate: Post-Hoc Control over Mixture-of-Experts to Address Shortcut Shifts in Natural Language Understanding
Transactions of the Association for Computational Linguistics, 2024, 12, pp. 1268-1289, with Ukyo Honda, Peinan Zhang, Masato Mita
Recent models for natural language understanding are inclined to exploit simple patterns in datasets, commonly known as shortcuts. These shortcuts hinge on spurious correlations between labels and latent features existing in the training data. At inference time, shortcut-dependent models are likely to generate erroneous predictions under distribution shifts, particularly when some latent features are no longer correlated with the labels. To avoid this, previous studies have trained models to eliminate the reliance on shortcuts. In this study, we explore a different direction: pessimistically aggregating the predictions of a mixture-of-experts, assuming each expert captures relatively different latent features. The experimental results demonstrate that our post-hoc control over the experts significantly enhances the model's robustness to the distribution shift in shortcuts. Additionally, we show that our approach has some practical advantages. We also analyze our model and provide results to support the assumption.
Keywords: Natural language understanding, Mixture ofExperts, Shortcuts, Spurious correlations
Bivariate Distribution Regression with Application to Insurance Data
Insurance: Mathematics and Economics, 2023, 113, pp. 215-232. with Yunyun Wang and Dan Zhu,
Understanding variable dependence, particularly eliciting their statistical properties given a set of covariates, provides the mathematical foundation in practical operations management such as risk analysis and decision-making given observed circumstances. This article presents an estimation method for modeling the conditional joint distribution of bivariate outcomes based on the distribution regression and factorization methods. This method is considered semiparametric in that it allows for flexible modeling of both the marginal and joint distributions conditional on covariates without imposing global parametric assumptions across the entire distribution. In contrast to existing parametric approaches, our method can accommodate discrete, continuous, or mixed variables, and provides a simple yet effective way to capture distributional dependence structures between bivariate outcomes and covariates. Various simulation results confirm that our method can perform similarly or better in finite samples compared to the alternative methods. In an application to the study of a motor third-party liability insurance portfolio, the proposed method effectively estimates risk measures such as the conditional Value-at-Risk and Expected Shortfall. This result suggests that this semiparametric approach can serve as an alternative in insurance risk management.
Keywords: Finance, Multivariate statistics, Risk management, Distribution Regression
Heterogeneous Impact of the Minimum Wage: Implications for Changes in Between- and Within-Group Inequality
Journal of Human Resources, 2023, 58(1), pp. 335-362, with Ken Yamada
Most of the workers who earn at or below the minimum wage are either less educated, young, or female in the United States. We examine the extent to which the minimum wage influences the wage differential among workers with different observed characteristics and the wage differential among workers with the same observed characteristics. Our results suggest that changes in the real value of the minimum wage account in part for the patterns of changes in education, experience, and gender wage differentials and for most of the changes in within-group wage differentials for workers with lower levels of experience.
COVID-19 poses a severe threat worldwide. This study analyzes its propagation and evaluates statistically the effect of mobility restriction policies on the spread of the disease.
We apply a variation of the stochastic Susceptible-Infectious-Recovered model to describe the temporal-spatial evolution of the disease across 33 provincial regions in China, where the disease was first identified. We employ Bayesian Markov Chain Monte-Carlo methods to estimate the model and to characterize a dynamic transmission network, which enables us to evaluate the effectiveness of various local and national policies. The spread of the disease in China was predominantly driven by community transmission within regions, which dropped substantially after local governments imposed various lockdown policies. Further, Hubei was only the epicenter of the early epidemic stage. Secondary epicenters, such as Beijing and Guangdong, had already become established by late January 2020. The transmission from these epicenters substantially declined following the introduction of mobility restrictions across regions. Conclusions: The spatial transmission network is able to differentiate the effect of the local lockdown policies and the cross-region mobility restrictions. We conclude that both are important policy tools for curbing the disease transmission. The coordination between central and local governments is important in suppressing the spread of infectious diseases.
Indirect inference requires simulating realizations of endogenous variables from the model under study. When the endogenous variables are discontinuous functions of the model parameters, the resulting indirect inference criterion function is discontinuous and does not permit the use of derivative-based optimization routines. Using a change of variables technique, we propose a novel simulation algorithm that alleviates the discontinuities inherent in such indirect inference criterion functions, and permits the application of derivative-based optimization routines to estimate the unknown model parameters. Unlike competing approaches, this approach does not rely on kernel smoothing or bandwidth parameters. Several Monte Carlo examples that have featured in the literature on indirect inference with discontinuous outcomes illustrate the approach, and demonstrate the superior performance of this approach over existing alternatives.
This paper shows that the Conditional Quantile Treatment Effect on the Treated is identified under (i) a Conditional Distributional Difference in Differences assumption and (ii) a new assumption that the dependence (the copula) between the change in untreated potential outcomes and the initial level of untreated potential outcomes is the same for the treated group and untreated group. We consider estimation and inference with discrete covariates and propose a uniform inference procedure based on the exchangeable bootstrap. Finally, we estimate the effect of increasing the minimum wage on the distribution of earnings for subgroups defined by race, gender, and education.
The issue addressed in this paper is that of testing for common breaks across or within equations of a multivariate system. Our framework is very general and allows integrated regressors and trends as well as stationary regressors. The null hypothesis is that breaks in different parameters occur at common locations and are separated by some positive fraction of the sample size unless they occur across different equations. Under the alternative hypothesis, the break dates across parameters are not the same and also need not be separated by a positive fraction of the sample size whether within or across equations. The test considered is the quasi-likelihood ratio test assuming normal errors, though as usual the limit distribution of the test remains valid with non-normal errors. Of independent interest, we provide results about the rate of convergence of the estimates when searching over all possible partitions subject only to the requirement that each regime contains at least as many observations as some positive fraction of the sample size, allowing break dates not separated by a positive fraction of the sample size across equations. Simulations show that the test has good finite sample properties. We also provide an application to issues related to level shifts and persistence for various measures of inflation to illustrate its usefulness.
This paper proposes the cross-quantilogram to measure the quantile dependence between two time series. We apply it to test the hypothesis that one time series has no directional predictability to another time series. We establish the asymptotic distribution of the cross-quantilogram and the corresponding test statistic. The limiting distributions depend on nuisance parameters. To construct consistent confidence intervals we employ a stationary bootstrap procedure; we establish consistency of this bootstrap. Also, we consider a self-normalized approach, which yields an asymptotically pivotal statistic under the null hypothesis of no predictability. We provide simulation studies and two empirical applications. First, we use the cross-quantilogram to detect predictability from stock variance to excess stock return. Compared to existing tools used in the literature of stock return predictability, our method provides a more complete relationship between a predictor and stock return. Second, we investigate the systemic risk of individual financial institutions, such as JP Morgan Chase, Morgan Stanley and AIG.
This paper studies identification and estimation of a censored quantile regression model for short panel data with fixed effects. Using the redistribution-of-mass idea, we obtain bounds on the conditional distribution of differences of the model across periods, under conditional quantile restrictions together with a weak conditional independence assumption along the lines of Rosen (2012). The inversion of the distribution bounds characterizes the sharp identified set via a set of inequalities based on conditional quantile functions. Due to the presence of censoring, some of the inequalities defining the identified set hold trivially and have no identification power. Moreover, those trivial inequalities cause a difficulty in estimating the identified set. To deal with the issue, we propose a two-step estimation method, where the first step consists of excluding trivial inequalities and the second step performs minimization of a convex criterion function using the remaining inequalities. We establish asymptotic properties of the set estimator and also consider sufficient conditions under which point identification can be attained.
This paper estimates the effects of unilateral divorce laws on divorce rates in the USA from a panel of state-level divorce rates. We use the interactive fixed-effects model to address the issue of endogeneity due to the association between cross-state unobserved heterogeneity and divorce law reforms. We document that earlier studies in the literature do not fully control for unobserved heterogeneity and result in mixed empirical evidence on the effects of divorce law reforms. While reconciling these conflicting results, our results suggest that divorce law reforms have temporal positive effects on divorce rates, thus confirming the 2006 findings of Wolfers. Via simulation experiments, we assess the degree to which faulty inclusion or faulty exclusion of interactive fixed effects affects the policy effect estimators. Our results suggest that faulty inclusion only results in efficiency loss whereas faulty exclusion causes bias.
This paper considers the estimation of multiple structural changes occurring at unknown dates in one or multiple conditional quantile functions. The analysis covers time series models as well as models with repeated cross-sections. We estimate the break dates and other parameters jointly by minimizing the check function over all permissible break dates. The limiting distribution of the estimator is derived and the coverage property of the resulting confidence interval is assessed via simulations. A procedure to determine the number of breaks is also discussed. Empirical applications to the quarterly US real GDP growth rate and the underage drunk driving data suggest that the method can deliver more informative results than the analysis of the conditional mean function alone.
Over the last decade, juvenile crime has become a serious social problem in Japan. The Juvenile Law was revised in 2001 to impose harsher punishment on juvenile offenders. This revision makes it possible to impose criminal punishment on 14- and 15-year-old criminal offenders, while those offenders aged 16–19 have always faced criminal punishment, both before and after the revision. Using this revision as a natural experiment, this study conducts a difference-in-differences estimation to examine the effect of punishment on juvenile crime. The analysis provides evidence that punishment can deter juvenile crime. In addition, this research examines the criminal behaviour of 13-year-olds, who face no change in punishment, but who soon will in the near future. The results suggest that the revision also had a negative impact on the criminal behaviour of these younger offenders.
We study the estimation of distributional treatment effects in randomized experiments with imperfect compliance. When participants do not adhere to their assigned treatments, we leverage treatment assignment as an instrumental variable to identify the local distributional treatment effect—the difference in outcome distributions between treatment and control groups for the subpopulation of compliers. We propose a regression-adjusted estimator based on a distribution regression framework with Neyman-orthogonal moment conditions, enabling robustness and flexibility with high-dimensional covariates. Our approach accommodates continuous, discrete, and mixed discrete-continuous outcomes, and applies under a broad class of covariate-adaptive randomization schemes, including stratified block designs and simple random sampling. We derive the estimator’s asymptotic distribution and show that it achieves the semiparametric efficiency bound. Simulation results demonstrate favorable finite-sample performance, and we demonstrate the method’s practical relevance in an application to the Oregon Health Insurance Experiment.
In-context learning (ICL) has emerged as a successful paradigm for leveraging large language models (LLMs). However, it often struggles to generalize beyond the distribution of the provided demonstrations. A recent advancement in enhancing robustness is ICL with explanations (X-ICL), which improves prediction reliability by guiding LLMs to understand and articulate the reasoning behind correct labels. Building on this approach, we introduce an advanced framework that extends X-ICL by systematically exploring explanations for all possible labels (X2-ICL), thereby enabling more comprehensive and robust decision-making. Experimental results on multiple natural language understanding datasets validate the effectiveness of X2-ICL, demonstrating significantly improved robustness to out-of-distribution data compared to the existing ICL approaches.
Keywords:
In-context learning, Large Language Models, Out-of-Distribution Data, Natural Language Processing, Spurious Correlation, Latent Variables, Generative artificial intelligence
On Efficient Estimation of Distributional Treatment Effects under Covariate-Adaptive Randomization
International Conference on Machine Learning (ICML), 2025, with Undral Byambadalai, Tomu Hirata, Shota Yasui
This paper focuses on the estimation of distributional treatment effects in randomized experiments that use covariate-adaptive randomization (CAR). These include designs such as Efron's biased-coin design and stratified block randomization, where participants are first grouped into strata based on baseline covariates and assigned treatments within each stratum to ensure balance across groups. In practice, datasets often contain additional covariates beyond the strata indicators. We propose a flexible distribution regression framework that leverages off-the-shelf machine learning methods to incorporate these additional covariates, enhancing the precision of distributional treatment effect estimates. We establish the asymptotic distribution of the proposed estimator and introduce a valid inference procedure. Furthermore, we derive the semiparametric efficiency bound for distributional treatment effects under CAR and demonstrate that our regression adjusted estimator attains this bound. Simulation studies and empirical analyses of microcredit programs highlight the practical advantages of our method.
We propose a novel regression adjustment method designed for estimating distributional treatment effect parameters in randomized experiments. Randomized experiments have been extensively used to estimate treatment effects in various scientific fields. However, to gain deeper insights, it is essential to estimate distributional treatment effects rather than relying solely on average effects. Our approach incorporates pre-treatment covariates into a distributional regression framework, utilizing machine learning techniques to improve the precision of distributional treatment effect estimators. The proposed approach can be readily implemented with off-the-shelf machine learning methods and remains valid as long as the nuisance components are reasonably well estimated. Also, we establish the asymptotic properties of the proposed estimator and present a uniformly valid inference method. Through simulation results and real data analysis, we demonstrate the effectiveness of integrating machine learning techniques in reducing the variance of distributional treatment effect estimators in finite samples.
Excellent tail performance is crucial for modern machine learning tasks, such as algorithmic fairness, class imbalance, and risk-sensitive decision making, as it ensures the effective handling of challenging samples within a dataset. Tail performance is also a vital determinant of success for personalized recommender systems to reduce the risk of losing users with low satisfaction. This study introduces a ``safe'' collaborative filtering method that prioritizes recommendation quality for less-satisfied users rather than focusing on the average performance. Our approach minimizes the conditional value at risk (CVaR), which represents the average risk over the tails of users' loss. To overcome computational challenges for web-scale recommender systems, we develop a robust yet practical algorithm that extends the most scalable method, implicit alternating least squares (iALS). Empirical evaluation on real-world datasets demonstrates the excellent tail performance of our approach while maintaining competitive computational efficiency.
Keywords:
Recommendation, Conditional Value at Risk, Matrix Factorization, Covolution
