Causal Inference Foundations

12 applied causal inference primitives for impact attribution and experiment design, backed by a formal theory map. Each primitive solves a specific identification or estimation problem. Primitives are domain-agnostic: the same instrumental-variable logic that handles omitted-variable bias in econometrics handles it in product analytics; the same difference-in-differences framework that evaluates policy interventions evaluates feature rollouts.

When to Apply

Apply causal-inference when:

• "Did the change cause the outcome, or just correlate?" question
• A/B test is impossible (rollout already happened, ethics, ramping risk) — observational methods needed
• Confounding suspected — non-random treatment assignment
• Heterogeneous treatment effects matter (CATE, uplift)
• Mediation question — "is the effect through path X or path Y?"
• Units interfere — marketplace, social graph, shared inventory, ranking model, or agents sharing a backend resource; randomization alone does not identify the launch effect
• LLM evaluation pipeline uses logged data — prompt distribution, judge bias, or user self-selection confound the quality signal (Pearl's Ladder applies: estimating P(Y|do(prompt)) is different from P(Y|prompt))

Skip and use simpler alternatives when:

• Clean RCT / A/B test is already running and units do not interfere — read the result, don't re-derive it observationally. If units share a marketplace, graph, or backend resource, the test is not clean: see Interference and SUTVA
• Question requests an associational summary only — descriptive analytics is enough. The magnitude of a causal effect still requires identification
• No plausible causal mechanism — correlation is just measurement, not insight
• Effective sample size or treatment overlap is inadequate for the target estimand — narrow the population, change the design, or collect more data
• Sensitivity analysis shows that a substantively plausible omitted confounder could reverse the decision — qualify the claim or use a stronger design
• Question is about strategic interaction (multi-actor) — use foundations-game-theory

Contents

• Quick Reference
• Primitive Index
• Formal Supporting Theory
• Anti-Patterns
• Misuse Boundaries
• Decision Checklist
• Composition Recipes
• Interference and SUTVA
• Expert Judgment
• Workflow
• ASCII Flow
• Related Skills
• Navigation
• Fact-Checking

Quick Reference

Primitive Index

Each primitive is summarized here, expanded in references/primitives-overview.md, and covered by standalone playbooks under assets/templates/causal-inference/. Use references/formal-theory-map.md when the task needs identification assumptions, estimand distinctions, or design boundaries.

Formal Supporting Theory

Anti-Patterns

Misuse Boundaries

Decision Checklist

Use this to pick the right method before modeling:

• Can you draw the assumed DAG? If not, stop — assumptions are implicit and untestable. Draw DAG (#1) first.
• Is the effect you want interventional (do(X)) or conditional? If interventional, check identifiability with do-calculus (#2).
• Can one unit's treatment change another unit's outcome? (marketplace supply/demand, social graph, shared inventory, ranking model, geographic proximity) If yes, specify direct, indirect, total or global estimand, exposure mapping and assignment support before choosing an estimator; redesign only when current support/assumptions do not identify the target. See Interference and SUTVA.
• Do you have an RCT or clean natural experiment? If yes, use the design directly. If no, continue.
• Is there a threshold that determines treatment? → RDD (#5).
• Is there pre/post data with a comparable untreated group? → DiD (#6). Check parallel trends first.
  • Is treatment staggered (units adopt at different times)? → Use Callaway–Sant'Anna, Sun–Abraham, BJS imputation, or Gardner 2-stage (see primitives-overview Primitive 6). Do NOT use plain TWFE — negative-weight bias.
  • Is parallel trends uncertain? → Apply HonestDiD (Rambachan & Roth 2023) for honest CIs under bounded violations.
• Pre/post data with donor pool but parallel trends uncertain? → Synthetic DiD (Arkhangelsky et al. 2021, #7 extension). Bridges SC and DiD.
• Single treated unit with no clean control? → Synthetic control (#7).
• Are there unobserved confounders and a valid instrument? → IV (#4). Validate exclusion restriction and check first-stage F.
• Observational data with measured confounders only? → Propensity score matching / IPW / DR (#8). Check overlap.
• Do you need individual-level or subgroup effect estimates? → CATE / uplift (#9). Choose meta-learner by sample size.
• Does the aggregate trend contradict subgroup evidence? → Check for Simpson's paradox via DAG stratification (#10).
• Is the total effect mediated by an intermediate variable? → Mediation analysis (#11). Requires no unmeasured exposure-mediator confounders.
• Is the conclusion actionable under unobserved confounding? → Select a design- and scale-compatible sensitivity method (#12), name plausible confounders, and report it.

Composition Recipes

Uplift from Observational Data

Objective: estimate individual-level treatment effects without an RCT.

Stack:

1. DAG (#1) — draw the assumed data-generating process; identify confounders.
2. Propensity score + doubly robust estimator (#8) — balance covariates; estimate the ATE under identification, overlap and nuisance-model conditions; double robustness is a consistency property, not unconditional finite-sample unbiasedness. When treatment is continuous (dosage, spend, exposure level), use kernel-based DML for the average dose-response function — Colangelo & Lee (2025, JBES).
3. CATE / X-learner (#9) — estimate heterogeneous effects using the debiased residuals.
4. Sensitivity analysis (#12) — use outcome-scale-compatible sensitivity for the strongest subgroup claim; use an E-value only for a risk-ratio estimand or a documented justified conversion. For DML/doubly robust pipelines, additionally apply OVB bounds via Chernozhukov et al. (2026, REStat) to assess robustness of the ATE claim.

Worked example: 50 k users; 15 k treated by a 20%-off discount (self-selected). Propensity model (logistic, 12 covariates) yields p̂ ∈ [0.05, 0.95] for 91% of treated; 9% is trimmed. DR-ATE = +$2.40/user (SE $0.31, 95% CI [$1.79, $3.01]). X-learner surfaces a high-value segment (top quintile by LTV) with CATE = +$4.10 (SE $0.52). These dollar mean differences do not supply the risk ratio required for an E-value. Use partial-R2/OVB sensitivity against named plausible confounders, and label the post-trimming population separately from the original user population. All numbers in this example are hypothetical.

When to add IV (#4): a valid instrument exists (e.g., randomized discount assignment, geographic variation); use it instead of propensity methods for the first-stage.

Policy Evaluation with No Control Group

Objective: estimate the impact of a policy or feature applied to a single market or cohort.

Stack:

1. DAG (#1) — map treatment, outcomes, and potential confounders over time.
2. Synthetic control (#7) — construct a weighted donor pool to serve as the counterfactual.
3. DiD robustness check (#6) — apply DiD on the synthetic control residual to quantify pre-trend fit.
4. Sensitivity analysis (#12) — donor leave-one-out, pre-fit and time-window sensitivity, with placebo/rank inference under explicitly justified exchangeability assumptions. Rosenbaum matched-assignment bounds do not directly apply to donor permutations.

Mechanism Attribution (Why Did the Effect Happen?)

Objective: decompose a total causal effect into direct and indirect (mediated) components.

Stack:

1. DAG (#1) — identify the mediator path; confirm no unmeasured exposure-mediator confounders.
2. Backdoor criterion (#3) — determine the adjustment set for total effect identification.
3. Propensity / DR estimator (#8) — produce balanced outcome estimates for mediation.
4. Mediation analysis (#11) — decompose NDE and NIE; report proportion mediated.
5. Sensitivity analysis (#12) — mediation-specific sensitivity for mediator-outcome confounding on the declared effect scale; an ordinary total-effect E-value is not automatically an indirect-effect sensitivity analysis.

LLM Evaluation Pipeline — Deconfounding the Quality Signal

Objective: estimate the causal effect of a prompt change, model update, or RLHF policy on output quality, when evaluation data are logged (non-randomised) and judge scores are potentially biased.

Context: LLM development pipelines generate observational logs. User prompt distribution, conversation history, judge LLM identity, and user self-selection all confound quality metrics. Simply comparing average scores before and after a model update conflates the treatment effect with distributional shift. (Reference: arxiv 2605.25998, "Causal Methods for LLM Development and Evaluation", May 2026.)

Stack:

1. DAG (#1) — draw: Prompt → LLM_response → Quality_score; annotate confounders (prompt difficulty, user type, judge identity) and potential colliders (filtered output).
2. Do-calculus / backdoor (#2, #3) — check whether P(Quality | do(model_update)) is identified given available logs; identify the minimal adjustment set.
3. Propensity / DR estimator (#8) — balance on prompt covariates and user context; use doubly robust ATE. For continuous interventions (e.g., RLHF reward weight), use kernel-based DML (Colangelo & Lee 2025).
4. CATE (#9) — surface heterogeneous effects by prompt category, task type, or user cohort; avoid reporting a flat ATE that masks regressions in a subgroup.
5. Sensitivity analysis (#12) — choose OVB/partial-R2 or other scale-compatible sensitivity for continuous quality scores; use an E-value only if quality is modeled as a risk ratio or a justified conversion is documented. Benchmark named judge-bias confounders.

Note on LLM-assisted causal discovery: LLMs can propose DAG edges from domain knowledge but cannot replace data-driven identification checks — autoregressive next-token modeling has no mechanism for establishing direction. Use LLM outputs as priors to seed a DAG; validate edges with statistical tests (faithfulness, independence). Do not treat LLM-generated graphs as identified causal models. The restriction is on decisional authority, not on all LLM involvement: LLM-guided heuristic search over the structure space is a legitimate accelerator, since the search result is still validated against data. Reported LLM causal-discovery accuracy is separately confounded by memorization: the standard bnlearn benchmark graphs (Sachs, Asia, Alarm, Child) are widely published and plausibly in pretraining corpora, so benchmark scores are weak evidence of causal reasoning — prefer a graph your own domain generated. (Wu, Yu, Wu & Tan 2025, arXiv:2506.00844; contamination caveat per CausalBench, arXiv:2404.06349.)

Interference and SUTVA: When Randomization Is Not Enough

No-interference is part of SUTVA, which also concerns treatment versions. Many conventional unit-level formulas assume it, but DAGs/SCMs and potential-outcome models can explicitly include cross-unit causes. Interference does not make every estimand unrecoverable. Specify the exposure mapping, the direct/indirect/total/global intervention contrast, assignment probabilities and support, consistency of exposures, and remaining identification assumptions. Randomization alone does not turn a direct effect into the all-treated-versus-all-control launch effect.

Counterexample: independent Bernoulli(.5) assignments Z1,Z2 with Y1=3Z1+7Z2 and Y2=3Z2+7Z1 have interference. The Horvitz–Thompson direct-effect estimator averaging 2ZiYi−2(1−Zi)Yi over the two units has expectation 3 across the four equally likely assignments. The global all-treated-versus-all-control contrast is 10. Both contrasts exist; they answer different questions. Exposure-probability weighting requires positive probability for the exposures being contrasted. Aronow & Samii (2017), design/exposure/estimand framework.

Identify the interference structure first, then pick the design:

The reporting distinction that matters: under interference, the unit-level "treatment effect" and the effect of switching everyone (the global/total treatment effect) are different quantities. Cluster or switchback designs target contrasts defined by their allocation, carryover and exposure assumptions; neither automatically identifies the global rollout effect. Say which contrast was identified and justify any extrapolation.

Agent and LLM products hit this directly: agents sharing a rate limit, a retrieval index, a cache, or a tool backend interfere through the shared resource, so per-session randomization can identify a different contrast and may understate or invert the launch effect; establish this for the actual resource/exposure model.

Expert Judgment

What separates an expert from a checklist-follower is not knowing more formulas — it is reading the shape of the data before picking a formula, and knowing which textbook assumption is the one that actually breaks.

Picking an Identification Strategy From Data Shape

• One treated unit, a time series, and a pool of comparable untreated units → synthetic control or synthetic DiD, not a hand-picked comparison unit. If pre-treatment fit is poor, say so and stop rather than force it.
• A rule with a hard numeric cutoff and enough density of units near it → RDD, not a linear control for the running variable. If the running variable is coarse (rounded scores, integer ages), check for heaping before trusting continuity.
• Treatment rolled out at different times across units → check whether never-treated or not-yet-treated units exist, then use a heterogeneity-robust staggered-DiD estimator (Callaway–Sant'Anna, Sun–Abraham, BJS, or Gardner). Plain TWFE is a bug, not a baseline, once adoption is staggered and effects can vary by cohort.
• Confounders you can name and measure completely, with common support across treated/control → propensity/DR. If you cannot name the confounders, no amount of covariate adjustment substitutes for a design — look for a natural experiment (IV, RDD) instead.
• An exogenous shock or rule that shifts treatment for some units and not others, for a reason unrelated to the outcome → IV, but only if the exclusion story survives being explained to a skeptical colleague in one sentence. If the one-sentence version needs three caveats, the instrument is probably not clean.
• The real question is "who benefits," not "what's the average effect" → CATE/uplift layered on top of an already-validated ATE/ATT, never as a substitute for identification. A confounded CATE just reports which subgroup has the most confounding.

The Assumption That Actually Fails in Practice

The textbook assumption is rarely violated the way the textbook describes it. What experts actually watch for:

Placebo and Robustness Checks an Expert Always Runs

• Placebo-in-time: rerun the design as if treatment happened one period earlier; expect a null effect.
• Placebo-in-space / placebo-outcome: rerun on units or outcomes the treatment should not affect.
• Leave-one-out: drop the highest-weight synthetic-control donor, or the strongest component of a composite instrument, and confirm the estimate does not collapse.
• Specification / bandwidth curve: show the estimate across a range of RDD bandwidths or DiD control sets, not just the one preferred specification.
• Randomization inference: use permutation p-values instead of asymptotic SEs when clusters or treated units are few (a handful of treated states or markets).
• Sensitivity analysis as a routine output, not an appendix: E-value, Rosenbaum bounds, or HonestDiD accompany every observational or DiD point estimate, not just the ones that look fragile.

When Causal ML Adds Nothing Over a Good Quasi-Experiment

• If a credible design already exists (valid IV, sharp RDD, staggered DiD with a heterogeneity-robust estimator) and the target is a single ATE/ATT/LATE, doubly-robust ML nuisance estimation buys efficiency, not identification. The design is doing the causal work; DML is just a better nuisance-function fitter.
• Causal ML (causal forests, DML, meta-learners) earns its complexity when: (a) covariates are high-dimensional with an unknown confounding functional form, (b) the question is heterogeneity (CATE/uplift) that a single quasi-experiment cannot answer without infeasible sample size, or (c) treatment is continuous/high-cardinality with no closed-form estimator.
• It does not repair a broken identification strategy. Running econml on top of a DiD with violated parallel trends, or a dowhy refutation suite on top of an IV with a leaky exclusion restriction, produces a precise, doubly-robust, wrong answer. Fix identification before reaching for machine learning.
• Where the literature is genuinely unsettled (state this plainly rather than picking a side): (1) which staggered-DiD estimator (Callaway–Sant'Anna, Sun–Abraham, BJS, Gardner) to prefer is setting-dependent, not resolved — the 2026 JEL practitioner's guide (Baker, Callaway, Cunningham, Goodman-Bacon & Sant'Anna) frames the choice by design and target estimand rather than naming a winner, and the estimators can disagree meaningfully on the same panel; (2) the best-practice sensitivity-analysis default for ML-based ATEs (Chernozhukov, Cinelli et al. 2026 vs. simpler partial-R² benchmarks) is still settling in applied practice; (3) using LLMs to propose or accelerate causal discovery has no consensus validation protocol as of mid-2026 — treat LLM-proposed edges as priors to test, not conclusions to report.

Workflow

1. Identify the causal question: intervention effect, mechanism, or heterogeneous effect?
2. Draw the DAG (#1). Identify confounders, mediators, and colliders.
3. Use the Decision Checklist to select the identification strategy.
4. Open references/primitives-overview.md for inputs, assumptions, and worked conceptual examples. Its Tooling Landscape table maps each primitive to the maintained Python/R library that implements it.
5. For multi-method stacks, use the Composition Recipes above.
6. Check references/patterns-scenarios-traps.md and always close with sensitivity analysis (#12) when reporting observational estimates. For risk ratios, python3 scripts/evalue.py --rr RR --ci-low LOW --ci-high HIGH validates the interval and uses the confidence bound closest to the null.

ASCII Flow

Causal claim or impact question
  -> State intervention, outcome, unit, and estimand
  -> Draw DAG and mark confounders, mediators, colliders
  -> Select identification strategy
     +-- randomized evidence exists -> analyze experiment directly
     +-- observational only -> choose IV, RDD, DiD, synthetic control, or propensity design
  -> Estimate effect and run sensitivity analysis
  -> Report assumptions, effect, uncertainty, and fragility

Related Skills

Wave 2 has landed. These consumer skills build applied-recipe layers on top of these primitives via their own references/causal-inference-applied.md (or domain-named equivalent) files. Each gates on this skill's When to Apply before invoking:

• ../marketing-cro/references/causal-inference-applied.md and causal-inference-experimentation.md — CUPED, switchback and geo-experiments, CATE for personalization
• ../marketing-product-analytics/references/causal-inference-applied.md and causal-inference-analytics.md — funnel and retention attribution
• ../marketing-paid-advertising/references/causal-inference-applied.md — media-mix and geo-lift attribution
• ../marketing-email-automation/references/causal-inference-applied.md — send-time and lifecycle-campaign lift
• ../data-analytics-engineering/references/causal-inference-applied.md — metric-layer causal contracts
• ../product-management/references/causal-inference-applied.md — feature-rollout impact
• ../startup-business-models/references/causal-inference-applied.md — pricing and monetization lift
• ../qa-debugging/references/causal-inference-applied.md — regression root-cause attribution

New consumer domain layers should follow the same gate-then-recipe pattern rather than duplicating the primitives themselves.

Navigation

• Practical completion contract and known-answer controls: references/practical-contract.md.

• Formal theory map: references/formal-theory-map.md

• Patterns, scenarios, and traps: references/patterns-scenarios-traps.md

• Full primitives overview with TOC: references/primitives-overview.md

• Per-primitive playbooks: assets/templates/causal-inference/README.md

• Sources: data/sources.json

Fact-Checking

• Pearl (2009) Causality is the canonical source for DAGs, do-calculus, and the backdoor/frontdoor criteria. Verify structural claims against that text.
• Imbens & Rubin (2015) Causal Inference for Statistics, Social, and Biomedical Sciences is the canonical source for potential-outcomes framework, IV, and matching.
• Angrist & Pischke (2009) Mostly Harmless Econometrics covers IV, RDD, and DiD in applied settings; use for identification assumption checks.
• Athey & Imbens (2017) machine-learning–based CATE estimation is the source for meta-learner claims; verify heterogeneous-effect benchmarks against that paper.
• Hernán & Robins What If (2020, freely available) is the canonical reference for time-varying treatments, IPW, and marginal structural models.
• Chernozhukov et al. (2018) on double/debiased machine learning (DML) is the source for doubly robust and Neyman-orthogonal estimator claims. For continuous treatments, the DML extension is Colangelo & Lee (2025, Journal of Business & Economic Statistics, doi:10.1080/07350015.2025.2505487). For omitted-variable sensitivity analysis of DML estimates, see Chernozhukov, Cinelli et al. (2026, Review of Economics and Statistics, doi:10.1162/REST.a.1705). Implemented in dml.sensemakr.
• Sensitivity analysis E-values: VanderWeele & Ding (2017); Rosenbaum bounds: Rosenbaum (2002) Observational Studies. Sensitivity analysis for IV estimates: Cinelli & Hazlett (2025, Biometrika, doi:10.1093/biomet/asaf004) extends the partial-R² OVB framework to handle exclusion-restriction violations and instrument confounding. Implemented in iv.sensemakr R package.
• For staggered DiD, the canonical method set (as of 2026) is: Callaway & Sant'Anna (2021, JoE, doi:10.1016/j.jeconom.2020.12.001); Sun & Abraham (2021, JoE, doi:10.1016/j.jeconom.2020.09.006); Borusyak, Jaravel & Spiess (2024, RES, doi:10.1093/restud/rhae011) imputation estimator; Gardner (2022, arXiv:2207.05943) two-stage DiD. Goodman-Bacon (2021, JoE, doi:10.1016/j.jeconom.2021.03.014) decomposition explains why plain TWFE fails. Navigational synthesis: Roth, Sant'Anna, Bilinski & Poe (2023, JoE, doi:10.1016/j.jeconom.2022.11.001). The current practitioner-facing reference is Baker, Callaway, Cunningham, Goodman-Bacon & Sant'Anna (2026, JEL 64(2), 498–557, doi:10.1257/jel.20251650) — organizes DiD designs by estimand, covariates, weights, and timing rather than prescribing one estimator.
• For parallel-trends robustness: Rambachan & Roth (2023, RES, doi:10.1093/restud/rhad018) HonestDiD. Pre-trend tests have low power; HonestDiD provides honest CIs under bounded violations without the binary pass/fail logic.
• For Synthetic DiD (bridging SC and DiD): Arkhangelsky et al. (2021, AER, doi:10.1257/aer.20190159). R package synthdid.
• For doubly robust DiD with covariates: Sant'Anna & Zhao (2020, JoE, doi:10.1016/j.jeconom.2020.06.003) DR-DiD; underpins the Callaway–Sant'Anna estimator.
• For interference and SUTVA violations: Bojinov, Simchi-Levi & Zhao (2023, Management Science 69(7), 3759–3777, doi:10.1287/mnsc.2022.4583) for optimal switchback design under carryover; Jia, Kallus & Yu (2025, arXiv:2312.15574) for clustered switchback under joint spatio-temporal interference; Peng, Ye & Zheng (2025, arXiv:2503.02271) for the differences-in-neighbors estimator under network interference. These are design methods — verify the assumed interference structure before citing an estimator's guarantees.
• Method effectiveness is sample-size and domain dependent. Validate identification assumptions explicitly before reporting estimates.
• Source links and verified dates in each per-primitive file are the canonical evidence tier.

Learnings Loop

When prior decisions or pitfalls are relevant, consult learnings.consolidated.md if present; use learnings.md only for needed history or as the available fallback. Otherwise skip both.

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.