Standard practice in the WordPress world is to stare at correlation-based metrics and hope. That habit burns development hours and tells you very little. If you want to know whether a bug fix or a new checkout flow actually moved revenue, you need Causal Inference Methods rather than another dashboard widget.
I have seen enough “successful” A/B tests that turned out to be seasonal noise, or a race condition in the analytics script, to stop trusting a raw dashboard. What I use now is a slower checklist that separates real impact from coincidence. If you have been fighting analytics monoliths, this is the layer that sits on top of that work.
1. Doubly robust estimation, or two chances to be right
Randomized trials are the ideal. On a live WooCommerce site users self-select instead: they join the loyalty program because they feel like it, and you cannot make them. The usual response is to model either the outcome with regression or the probability of joining with propensity scores. Get that one model slightly wrong and the estimate is biased.
Doubly Robust Estimation, also called Augmented Inverse Probability Weighting (AIPW), runs both models together. As long as either the outcome model or the propensity model is correct, the estimate stays consistent. Same reasoning as a redundant server: one of the two has to hold, not both.
# Simplified AIPW Logic using sklearn
import numpy as np
from sklearn.ensemble import GradientBoostingClassifier, GradientBoostingRegressor
def bbioon_doubly_robust(Y, T, X):
# T = Treatment (e.g., joined loyalty program)
# Y = Outcome (e.g., total spend)
# X = Covariates (e.g., age, previous spend)
# 1. Propensity Model (Who joins?)
ps_model = GradientBoostingClassifier().fit(X, T)
e = ps_model.predict_proba(X)[:, 1]
e = np.clip(e, 0.05, 0.95) # Clip to avoid division by zero
# 2. Outcome Models (What do they spend?)
mu1 = GradientBoostingRegressor().fit(X[T==1], Y[T==1]).predict(X)
mu0 = GradientBoostingRegressor().fit(X[T==0], Y[T==0]).predict(X)
# 3. Combine
dr_ate = np.mean((mu1 + T * (Y - mu1) / e) - (mu0 + (1 - T) * (Y - mu0) / (1 - e)))
return dr_ate2. Instrumental variables, when the confounder is unmeasured
Some of what drives the outcome never lands in a database. User motivation, for one. Instrumental Variables (IV) is the tool for that case: you find a nudge, the instrument, that changes whether someone takes the treatment without touching the outcome directly. It is the least intuitive of the Causal Inference Methods here and often the only one available.
Say you mailed random discount codes. The receipt of that email is your instrument. It pushes people toward a purchase without being the same thing as their underlying willingness to spend, so isolating the variation the email caused leaves you with a clean estimate for the compliers.
3. Regression discontinuity at a hard cutoff
Hard cutoffs do some of the work for you. Free shipping above $100, for example. A cart at $99.99 and a cart at $100.01 belong to the same kind of shopper, so comparing the two sides of that line gives you something close to a randomized experiment. Nobody games their way to one cent over a threshold, which is what makes the estimate credible.
On the tracking side, the WooCommerce performance updates cover how these metrics get recorded once the volume climbs.
4. Difference-in-differences with a staggered rollout
Roll a new checkout API out city by city and you have staggered adoption. Two-Way Fixed Effects, the old approach, breaks once the treatment effect changes over time. Current Causal Inference Methods use group-time specific effects instead, comparing newly treated units only against units not yet treated, so nothing gets measured against a city that switched over months ago.
5. Heterogeneous Treatment Effects (CATE)
An average effect can hide the thing you needed to know. A plugin that speeds up 90% of sites and crashes the other 10% still reports a positive average. Conditional Average Treatment Effects (CATE) tell you who sits in which group. EconML from Microsoft gives you Causal Forests, which pick out those segments for you.
from econml.dml import CausalForestDML
# X: Site Speed, Mobile vs Desktop, Region
# T: New Interactivity API Enabled
# Y: Time on Page
cf = CausalForestDML(model_y=GradientBoostingRegressor(), model_t=GradientBoostingClassifier())
cf.fit(Y, T, X=X)
ite = cf.effect(X) # Get effect for every individual userIf this kind of analysis is eating your week, I take it on for clients. I have been building WordPress sites since the 4.x days.
Where to start
None of this is really about the math. It is about not being fooled by your own numbers. Whether you are chasing a performance bottleneck or rebuilding a legacy analytics pipeline, the habit worth keeping is asking why the data looks the way it does before you act on it. statsmodels covers the baseline and EconML handles the rest.
