DGP Dossier: synth_hd_sparse_plm

1. Identity & Status

DGP ID	synth_hd_sparse_plm
Version	1.5.0
Status	experimental
Difficulty	★★★★☆ (4/5)
Stress Profile	overlap: moderatenoise: gaussianlinearity: lineareffect: constanttarget: both

2. What This DGP Stresses (Intent)

Sparsity and regularization in high dimensions. Tests whether estimators can recover a sparse linear signal without overfitting. Toeplitz correlation ((= 0.95)) increases collinearity and makes nuisance learning fragile, but overlap is restored to be moderate in v1.5.0 so the primary stress is high-dimensional sparsity rather than positivity.

3. Identification Assumptions (Explicit)

• Selection on observables holds.
• Overlap is moderate.
• SUTVA holds.

4. Mathematical Specification

Covariates: [ X ^{100},X (0, ),_{ij} = 0.95^{|i-j|} ]

Outcome (control): [ 0(X) = {j=1}^{5} X_j ]

Propensity: [ (p(X)) = 0.2_{j=1}^{5} X_j ]

Treatment effect: [ (X) ]

Noise: [ (0, 1^2) ]

5. Oracle Truth Definition

• True ATT is computed from structural_te among treated units.
• True QST is computed on the oracle tau grid (cs_get_oracle_qst()).

6. Visual Diagnostics (n = 5000)

7. Empirical Validation

lm_att serves as the baseline: it matches the sparse linear outcome model.

Victim: ipw_att uses standard (unregularized) logistic regression for the propensity model. In \(P=100\) with \(\rho=0.95\), collinearity and low N/P can still make nuisance estimation fragile, but v1.5.0 restores moderate overlap so the primary stress is high-dimensional sparsity rather than positivity.

estimator_id	mean_bias	rmse
ipw_att	-0.166	1.105
lm_att	0.032	0.118
oracle_att	0.000	0.000

8. Failure Mode Summary

• High-dimensional nuisance estimation can drown out the sparse signal.
• Non-parametric methods that fit complex nuisance surfaces can degrade sharply in 100D.

9. Implementation Reference

• Generator: R/dgp-synth-hd-sparse-plm.R
• Registry: cs_dgp_registry() entry for synth_hd_sparse_plm
• Oracle truth: cs_get_oracle_qst("synth_hd_sparse_plm", version = "1.5.0")

10. Validation Checklist

• Toeplitz covariance specified
• Sparse outcome/propensity specified
• Love plot replaces X1 vs X2 scatter

11. Changelog

• v1.4.0: Parameter hardening (Deprecated v1.3.0).
• v1.5.0: Reduced propensity coefficients to 0.2. v1.4.0 deprecated due to unintended positivity violation caused by correlation inflation.

Appendix: Implementation

Source code for: dgp_synth_hd_sparse_plm_v150

function (n, seed = NULL, include_truth = TRUE, oracle_only = FALSE) 
{
    if (!is.null(seed)) {
        cs_set_rng(seed)
    }
    p_hd <- 100L
    idx <- seq_len(p_hd)
    Sigma <- outer(idx, idx, function(i, j) 0.95^abs(i - j))
    L <- chol(Sigma)
    Z <- matrix(stats::rnorm(n * p_hd), nrow = n)
    X <- Z %*% L
    colnames(X) <- paste0("X", seq_len(p_hd))
    beta_y <- c(rep(1, 5), rep(0, p_hd - 5))
    mu0 <- as.numeric(X %*% beta_y)
    eps <- stats::rnorm(n, mean = 0, sd = 1)
    y0 <- mu0 + eps
    tau <- rep(1, n)
    y1 <- y0 + tau
    gamma <- c(0.2, 0.2, 0.2, 0.2, 0.2, rep(0, p_hd - 5))
    lin_ps <- as.numeric(X %*% gamma)
    p <- stats::plogis(lin_ps)
    w <- stats::rbinom(n, size = 1L, prob = p)
    y <- ifelse(w == 1L, y1, y0)
    true_att <- cs_true_att(structural_te = tau, w = w)
    true_qst <- if (isTRUE(include_truth)) {
        cs_get_oracle_qst("synth_hd_sparse_plm", version = "1.5.0")
    }
    else {
        tibble::tibble(tau = cs_tau_oracle, value = rep(NA_real_, 
            length(cs_tau_oracle)))
    }
    df <- tibble::tibble(y = y, w = w, y0 = y0, y1 = y1, p = p, 
        structural_te = tau)
    df <- dplyr::bind_cols(df, tibble::as_tibble(X))
    out <- list(df = df, true_att = true_att, true_qst = true_qst, 
        meta = list(dgp_id = "synth_hd_sparse_plm", version = "1.5.0", 
            type = "synthetic", params = list(n = n, seed = seed), 
            structural_te = tau))
    cs_check_dgp_synthetic(out)
    out
}
<bytecode: 0x556365ce5e90>
<environment: namespace:CausalStress>