When Events Change Samples: Disentangling Causal Effects from Compositional Bias in Quasi-Experimental Designs
Klara Müller klara.mueller@uni-mannheim.de
Version: MZES Social Science Data Lab, 2026-02-25
From political psychology, we know that salient political events can shape political attitudes and behavior. In studying such event effects, quasi-experimental designs, such as the unexpected event during survey design (UESD), have become increasingly popular. To allow for causal identification, these designs rest on quite strict assumptions, one of which is the random assignment of respondents to the pre- and post-event sample and the comparability of these samples. From survey methodology, however, we also know that external events can affect who responds to surveys. If event-triggered shifts in survey participation and sample composition are related to outcome variables of interest, causal estimates of an event’s effect may be biased. This problem is particularly challenging when compositional shifts involve unobserved or even unobservable factors. In this talk, I provide intuition for how such compositional bias can threaten causal conclusions in quasi-experimental settings. I present a framework to disentangle an event’s “true” causal effect from compositional bias, with a particular focus on the UESD approach. The framework outlines practical strategies to adjust for observable imbalances and extends sensitivity analyses to assess how strong unobserved confounders would have to be to change or undermine substantive causal conclusions. I illustrate the approach using the rally-around-the-flag effect following the 2015 Charlie Hebdo attacks in France, as well as replications of published UESD studies on terrorist events and rally-style outcomes. By addressing both observable and unobservable sources of bias, this framework enhances causal inference and strengthens the credibility of public opinion research in dynamic political contexts.
📝 Slides 1
Klara Müller is a PhD researcher at the University of Mannheim. Her research lies at the intersection of political psychology and quantitative methods. In particular, she focuses on how political contexts affect not only political behavior but also the quality of survey data, and what this implies for survey-based measurement and causal inference.