CausalStress

CausalStress v0.2.1 scientific status CausalStress has a tested execution and evidence contract, but most bundled DGPs have not completed scientific validation. Use it for development and governed benchmarking, not as evidence that an estimator is scientifically valid across the bundled scenarios.

CausalStress is an R framework for benchmarking causal-inference estimators against governed synthetic data-generating processes (DGPs). It separates estimator-visible data from runner-owned truth, records version and provenance metadata, and exposes typed score records for analysis.

Current scientific scope

Version 0.2.1 implements scoring for:

• ATT: average treatment effect on the treated;
• ATE: finite-sample average treatment effect;
• QST: quantile shift for the treated.

CATE is registered but not implemented. A CATE-only task fails closed; when CATE is included in a mixed request, it is represented by a target_not_implemented status row. This keeps the intended target explicit without claiming a working CATE benchmark.

All bundled DGPs are synthetic. Real-data DGP support is planned, but no real-data DGP or public DGP-registration API exists in v0.2.1.

The registry contains 12 DGP IDs and 24 immutable versioned entries. The current default stable versions are synth_baseline and synth_heavytail; the other ten DGP IDs are experimental and produce a governance warning when selected.

Heavy-tail boundary. synth_heavytail is intentionally a no-finite-mean regime. Its true_att is the governed finite-sample structural signal anchor mean(tau_i : W_i = 1), not a conventional superpopulation mean potential-outcome ATT, which does not exist under its Cauchy-mixture noise. Run ATT estimators there to diagnose breakdown, but do not use their mean error, RMSE, coverage, or ranking for an ATT shootout. Use QST for valid distributional comparisons.

DGP ID	Current status
synth_baseline	stable
synth_heavytail	stable
synth_placebo_tau0	experimental
synth_qte1	experimental
synth_nonlinear_heteroskedastic	experimental
synth_overlap_stressed	experimental
synth_tilt_mild	experimental
synth_placebo_nonlinear	experimental
synth_placebo_heavytail	experimental
synth_placebo_tilted	experimental
synth_placebo_kangschafer	experimental
synth_hd_sparse_plm	experimental

Passing the package contract and validation suite establishes deterministic execution, schema conformance, and truth separation. It does not replace human review of a DGP’s scientific interpretation, historical fidelity, or citations.

What the framework enforces

• Truth separation. Synthetic DGPs return observed data and runner-owned structural/distributional truth through separate contract fields.
• Airlock sanitation. Estimators invoked through the runner do not receive y0, y1, p, or structural_te unless an explicitly governed oracle grant permits a named truth column. Code that bypasses the runner is outside this guarantee.
• Deterministic task RNG. Each task is seeded at the runner boundary under the governed RNG contract.
• Versioned identity. Results carry DGP/estimator versions, fit identity, score-record identity, score-row identity, truth version, and configuration identity.
• Persistence and resume. Serial runs can be written to a pins board. Parallel persistence uses worker staging followed by controlled consolidation.

The governing rules and operational contracts live under inst/design/.

Installation

CausalStress is currently installed from GitHub:

# install.packages("pak")
pak::pak("blechturm/CausalStress")

Canonical workflow

cs_run_single() returns a structured result list. The typed scores table is the canonical scoring surface; att and qst remain compatibility projections.

library(CausalStress)
library(dplyr)

single <- cs_run_single(
  dgp_id = "synth_baseline",
  estimator_id = "lm_att",
  n = 500,
  seed = 1
)

names(single)
#> [1] "outputs"    "scores"     "att"        "qst"        "boot_draws"
#> [6] "meta"       "provenance"

cs_collect_scores(single) |>
  select(estimand_target_id, metric_id, estimate, truth, error) |>
  knitr::kable(digits = 3)

estimand_target_id	metric_id	estimate	truth	error
att	point_error	1.037	1.139	-0.102

For repeated benchmarking, run a grid, optionally persist each serial result, then summarize or audit it:

library(pins)

board <- pins::board_temp()

runs <- cs_run_grid(
  dgp_ids = "synth_baseline",
  estimator_ids = c("lm_att", "ipw_att"),
  n = 500,
  seeds = 1:3,
  board = board,
  show_progress = FALSE
)

cs_summarise_runs(runs) |>
  select(dgp_id, estimator_id, n_runs, mean_error, mean_abs_error) |>
  knitr::kable(digits = 3)

dgp_id	estimator_id	n_runs	mean_error	mean_abs_error
synth_baseline	ipw_att	3	-0.069	0.092
synth_baseline	lm_att	3	0.002	0.072

cs_audit(board) |>
  select(dgp_id, estimator_id, seed, estimator_version) |>
  head() |>
  knitr::kable()

dgp_id	estimator_id	seed	estimator_version
synth_baseline	ipw_att	1	0.2.1
synth_baseline	ipw_att	2	0.2.1
synth_baseline	ipw_att	3	0.2.1
synth_baseline	lm_att	1	0.2.1
synth_baseline	lm_att	2	0.2.1
synth_baseline	lm_att	3	0.2.1

For new analysis code, prefer:

scores <- cs_collect_scores(runs)

cs_collect_att() and cs_collect_qst() are retained as compatibility projections for existing ATT/QST workflows.

Estimators

The package registry contains eight estimators. The three core estimators need only the package’s hard dependencies; the remaining estimators require their listed optional packages.

Estimator ID	Target output	Dependency
oracle_att	ATT, ATE	core
lm_att	ATT	core
ipw_att	ATT	core
gengc	ATT, QST	GenGC
gengc_dr	ATT, QST	GenGC
grf_dr_att	ATT	grf
bart_att	ATT	bartCause
tmle_att	ATT	tmle, SuperLearner

User-defined estimators can be added with cs_register_estimator(). See the estimator extension guide for the executable contract, registration lifetime, Airlock boundary, dependency handling, and typed output shapes.

Experimental parallel execution

Parallel execution exists, but remains explicitly experimental. It requires both parallel = TRUE and experimental_parallel = TRUE, emits a warning, and records the backend/thread-cap provenance. When a board is supplied, a staging_dir is mandatory so workers do not write directly to pins.

future::plan(future::multisession, workers = 2)

parallel_runs <- cs_run_grid(
  dgp_ids = "synth_baseline",
  estimator_ids = "lm_att",
  n = 500,
  seeds = 1:20,
  board = board,
  parallel = TRUE,
  experimental_parallel = TRUE,
  staging_dir = tempfile("causalstress-staging-")
)

The explicit opt-in is a readiness boundary, not a production-safety claim.

Documentation scope

The Quarto/pkgdown site contains:

• the canonical typed-scoring workflow;
• the user-defined estimator contract;
• the native synthetic-DGP contribution guide;
• focused benchmarking, persistence, distributional, and audit guides; and
• one reproducible report for each registered DGP.

The DGP contribution guide describes maintainer-reviewed package development. It does not imply that a public runtime DGP-registration API, real-data DGPs, or user-defined DGP families already exist.

Citation

To obtain the citation for the installed package version:

citation("CausalStress")