The Comparison Everyone Wants To Make
The tempting move is always the same: treated users did better, so the treatment worked. Ship it, screenshot the dashboard, present it Monday. That sentence is also exactly how confounding wins every time nobody checks the math.
orthoshift creates a population where treatment depends on covariates, outcomes depend on the same covariates, and the true average treatment effect is known in advance. That's the only honest way to grade an estimator: against the answer key, not against a vibe.
Orthogonal Residuals Do The Boring, Important Work
The estimator fits nuisance models for outcome and treatment, residualizes both, then estimates the treatment effect from what's left over. That orthogonal step is the entire trick, and it's less exciting than it sounds, which is exactly why it works.
I kept the linear algebra in the repo instead of hiding it behind an import. No scikit-learn, no hidden solver, no black box you have to trust on faith. Open the ridge routine and watch the estimate get built in front of you.
The Number
The true effect is 1.986. Naive difference in means reports 3.798, missing by 1.812, which is confounding lying to your face with a completely straight expression. Orthogonal DML reports 1.864, missing by only 0.122.
Look at That Table Again
I want you to reread the number I just gave you. Orthogonal DML misses by 0.122. Adjusted OLS, the simpler method sitting one row above it in the exact same table, misses by 0.114. Closer. At the precise seed printed in this README, the fancy orthogonalized estimator is already behind the plain linear regression, and nobody, including me the first time I wrote this benchmark, seemed to notice.
So I checked whether that was a fluke of one random seed or an actual pattern. I ran both estimators across sixty different seeds. Plain adjusted OLS won the head to head against orthogonal DML forty two times out of sixty. I built a second, disjoint set of thirty seeds and evaluated it exactly once to make sure I wasn't fooling myself with the same numbers twice: OLS still came out ahead, sixteen wins to fourteen.
The reason turned out to be structural, not random. Both estimators lean on the identical feature function for their adjustment model, the same handful of polynomial terms, and that exact functional form happens to be a perfect match for the true outcome model I built the synthetic population from. Double machine learning's actual selling point is robustness when your adjustment model is wrong, when you can't write down the true relationship and have to lean on something flexible instead. I never tested that scenario. I handed both estimators the answer key.
None of this means the DML implementation is broken. It is a correct cross fitted orthogonal estimator, doing exactly what the math says it should. It means this particular benchmark, with a low dimensional confounder and an adjustment model that already matches the truth, is precisely the one regime where DML's extra machinery has nothing left to protect you from. The real, dramatic, completely uncontested result is still there: both methods crush the naive unadjusted comparison by roughly forty times over. That part of the story never needed defending.
The original data generator and estimators are untouched, and the published table above still reproduces exactly. 12 tests pass locally and on GitHub Actions across Python 3.9, 3.11, and 3.13.