Two failure modes in causalml/propensity.py defaults, both measured on IHDP replication 0 (672 rows, 18% treated, fetch_ihdp(replication=0, split="train")), master (18ebfb0).
1. ElasticNetPropensityModel and LogisticRegressionPropensityModel collapse to the treated share. With defaults, every score comes out 0.183 (sd 0.000; uncalibrated AUC 0.395). This is consistent with LogisticRegressionCV's default accuracy scoring being penalty-indifferent on imbalanced data — predicting the majority class scores identically for every penalty, so cross-validation selects a degenerate one. Passing a wider grid recovers a working model:
from causalml.propensity import ElasticNetPropensityModel
import numpy as np
ElasticNetPropensityModel().fit_predict(X, w) # constant 0.183
ElasticNetPropensityModel(Cs=np.logspace(0, 3, 8)).fit_predict(X, w) # AUC 0.761
Suggestion: score the internal CV with log-loss or AUC instead of accuracy, and/or widen the default Cs.
2. GradientBoostedPropensityModel overfits. In-sample AUC is 1.000 with defaults and 0.946 with early_stop=True, with scores saturating at the 0.001/0.999 clip bounds. Estimators that weight by inverse propensity are unstable with scores like these.
Suggestion: stronger default regularization or a documented warning that its output needs out-of-fold prediction.
These matter beyond direct callers: the meta-learners fall back to ElasticNetPropensityModel when p=None, so on data like IHDP the X-/R-/DR-learners silently run on a constant propensity. Grouped into one issue because both fixes live in propensity.py.
🤖 Generated with Claude Code
Two failure modes in
causalml/propensity.pydefaults, both measured on IHDP replication 0 (672 rows, 18% treated,fetch_ihdp(replication=0, split="train")), master (18ebfb0).1.
ElasticNetPropensityModelandLogisticRegressionPropensityModelcollapse to the treated share. With defaults, every score comes out 0.183 (sd 0.000; uncalibrated AUC 0.395). This is consistent withLogisticRegressionCV's default accuracy scoring being penalty-indifferent on imbalanced data — predicting the majority class scores identically for every penalty, so cross-validation selects a degenerate one. Passing a wider grid recovers a working model:Suggestion: score the internal CV with log-loss or AUC instead of accuracy, and/or widen the default
Cs.2.
GradientBoostedPropensityModeloverfits. In-sample AUC is 1.000 with defaults and 0.946 withearly_stop=True, with scores saturating at the 0.001/0.999 clip bounds. Estimators that weight by inverse propensity are unstable with scores like these.Suggestion: stronger default regularization or a documented warning that its output needs out-of-fold prediction.
These matter beyond direct callers: the meta-learners fall back to
ElasticNetPropensityModelwhenp=None, so on data like IHDP the X-/R-/DR-learners silently run on a constant propensity. Grouped into one issue because both fixes live inpropensity.py.🤖 Generated with Claude Code