Measuring ROI on a WordPress store usually comes down to one habit: look at the graph in Google Analytics, and if it goes up, keep doing it. Leaning on raw correlation like that is a disaster. Businesses burn thousands on campaigns that never caused the growth they are taking credit for. Sorting out which ones did starts with propensity score matching.
The correlation trap in e-commerce analytics
I have watched too many clients celebrate a “successful” campaign because their best customers used a coupon. Those customers were going to buy anyway. Compare the average spend of people who saw an ad against people who did not and you are measuring pre-existing bias, not campaign impact. Your high-value users are more likely to engage with the site in the first place, so the data is skewed before the campaign even runs.
This is where data modeling lets you down. Reports that aggregate raw numbers without accounting for how customers behave are fiction. I have written before about why your analytics reports are slow and wrong, and the fix there was not a faster server either. It was a better statistical approach.
How propensity score matching gets closer to the truth
Propensity score matching (PSM) levels the playing field by finding “statistical twins” in your data. For every customer who got the treatment, say seeing an ad, the algorithm looks for a nearly identical customer who did not. Comparing those matched pairs isolates what the campaign actually did. It also handles the confounding variables, age, past spend, device type, that otherwise muddy the water.
Step 1: calculate the propensity scores
Start with a logistic regression model. You are not predicting the future here. You want the probability, the propensity, that a user would take a specific action given their history. That is what balances the groups so you are not comparing apples to oranges.
import pandas as pd
from sklearn.linear_model import LogisticRegression
# Define your covariates (features) and the treatment flag
covariates = ['age', 'past_spend', 'is_mobile']
treatment_col = 'saw_ad'
# Calculate Propensity Scores
lr = LogisticRegression()
X = df[covariates]
y = df[treatment_col]
lr.fit(X, y)
# Store the probability of being in the Treatment group
df['pscore'] = lr.predict_proba(X)[:, 1]Step 2: find your statistical twins
Then a nearest neighbors algorithm pairs each treated user with a control user carrying a similar score. Distance matters here: if a match sits too far away, throw it out. A small clean sample beats a big noisy one. The Scikit-learn NearestNeighbors documentation covers the math behind the comparison.
from sklearn.neighbors import NearestNeighbors
# Filter into treatment and control groups
treated = df[df[treatment_col] == 1].copy()
control = df[df[treatment_col] == 0].copy()
# Match using pscore and age (calibration)
caliper = 0.05
nn = NearestNeighbors(n_neighbors=1, radius=caliper)
nn.fit(control[['pscore', 'age']])
# Find the pairs
distances, indices = nn.kneighbors(treated[['pscore', 'age']])Measuring the real impact
With the matched pairs in hand you can run your T-tests and work out effect size (Cohen’s D). More often than not, the “13% increase in spend” from the raw dashboard turns out to be statistically insignificant once selection bias is accounted for. Marketing teams hate hearing it, and it is still the only way to stop pouring budget into nothing. I went through the same trap in my guide on interpreting A/B test results correctly.
If this kind of analysis is eating your dev hours, hand it over. I have been wrestling with WordPress and custom data analysis since the 4.x days.
Match first, then decide
Vibe-based analytics stop working once a WooCommerce store gets big enough, and simple averages are where they break. PSM is not reserved for data scientists at big tech firms. Any developer who has to report an ROI number and stand behind it can run it. Rework your analytics before another quarter’s budget disappears into noise.
