Machine learning predictions can call most World Cup matches before kickoff, and they almost never call a draw. That is the short version of what Marco Hening Tallarico found when he built a football forecaster in R from roughly 49,000 international matches going back to 1872. It is the part worth reading even if you will never forecast a match.
His final LightGBM model identified about 87 percent of actual home wins and about 63 percent of away wins on the held-out test set. Draws came in at 2 correct out of 1,784. I have no interest in selling football forecasts, but I care a lot about why the middle class of a three-way outcome refuses to be predicted, because that shape shows up in store data constantly.
Can machine learning predictions handle the draw?
About 22 percent of international matches end level, by his count, so a fifth of the target is a class the model has to pick sometimes. The baseline multinomial regression never once picked draw as the top class. The confusion matrix has no draw column at all. That failure is not specific to one algorithm: when he isolated confident wrong predictions, most were actual draws that the models had called as home wins with probability above 0.60.
He then engineered features aimed straight at draws: the absolute Elo gap between the teams, each team’s draw rate over the previous 5 and 10 matches, combined form draw rates, plus context flags for neutral ground, World Cup matches and friendlies. True-positive draws rose 3.3 percentage points. That is still nowhere near a fifth of outcomes, and he says so plainly.
The calibration plots are the honest part. Home-win and away-win probabilities track observed rates closely, so those numbers behave like real probabilities rather than scores. Draw probabilities are calibrated too, but inside a narrow band. In the most evenly matched fixtures the observed draw rate is roughly one third while the models assign about one quarter. The direction is right and the magnitude is short, so under the argmax rule, where you take the class with the highest probability, a draw almost never comes out on top.
This is the failure I would watch for on a WooCommerce project. Say the classes are will reorder, undecided, will churn. The undecided customers are your draws. If the interface shows a badge from the top class only, the model will badge strong buyers and lost buyers and say nothing about the group in the middle, which is often the group a campaign could still reach. Show the distribution, or at least the top two probabilities, and read up on why aggregate accuracy hides this kind of hole before you promise anything.
Where does the data leak in a project like this?
Two places, and both are plumbing rather than modeling. First the join. Match results and Elo ratings live in separate files and the team names do not match across 336 teams, so every name gets normalized to a canonical key before joining. His R version strips accents, lowercases and removes anything that is not a letter or digit. Doing the same job on two WordPress exports, I would reach for this:
<?php
function bbioon_join_key( string $raw ) : string {
$key = remove_accents( $raw ); // WP core, handles Côte d'Ivoire
$key = strtolower( $key );
$key = preg_replace( '/[^a-z0-9]+/', '', $key );
return trim( $key );
}remove_accents is a WordPress core function, which is why this belongs in a plugin instead of a hand-rolled transliteration table. The second leak is sneakier. Elo ratings update after the match, so attaching the same-day rating to a match hands the model a number that already contains the result. He uses the most recent pre-match rating and keeps the age of that rating in days as its own feature, on the theory that a fresh rating carries more signal than a stale one. A convenience like same-day ratings looks harmless right up until your validation numbers are suspiciously good.
The split matters just as much. Training uses pre-2018 data, validation takes the latest slice of that pool, and the test set is everything from 2018 onward, about 8,000 matches from the CC0 dataset at martj42/international_results. A random split would have averaged 2024 into a model being asked about 2016. Whenever your rows carry a timestamp, split on time, or you are grading the model on information it would not have had.
This plumbing is the part of a prediction feature I usually end up owning for clients: getting two sources joined under one key, the split honest, and the output rendered so the middle class stays visible. If you have a prediction feature on a WordPress or WooCommerce roadmap and want the pipeline pressure-tested before it ships, that is work I take on.
Should you ship the gradient boosted model?
His own numbers argue against his own pick, and he admits it. The selected LightGBM model scored 0.893 validation log loss and 0.873 on test. Multinomial regression trailed by about 0.002 on validation, then slightly beat LightGBM on both log loss and macro F1 on the test set. Goalscorer features did not move log loss at all. Feature importance puts rating difference first and neutral ground a distant second, so most of the signal sat in two columns that existed before any tuning.
LightGBM itself is a fine library and the parameter care in the article is real: learning rate, leaves, depth, minimum data per leaf, both regularization terms, feature and row fractions. But when the margin over an interpretable regression is that thin and flips on the held-out set, my default is to ship the regression. Whoever maintains it next year has to explain to a store owner why the model says what it says, and coefficients times inputs is an answer they can give. A 0.002 margin is not much of a prize for the extra maintenance. (I have not checked whether this holds for ordered outcomes like time to first purchase, where boosting often earns a bigger gap than it did here.)
The same logic applies to features as to models. His compact set of lagged form features helped on validation and barely moved the test set, and the goalscorer work added nothing measurable. Feature work that pays 0.002 is still work you have to maintain forever. Keep that in mind the next time a small data machine learning project on a WooCommerce store tempts you toward one more derived column.
Before promising anyone a prediction feature, run the cheap test. Take the historical rows, split on time, and look at per-class recall and calibration instead of aggregate accuracy. If recall on the middle class is near zero, you have a two-class predictor wearing a three-class label, and you should say so before the client builds a screen around it. The World Cup data says that is what to expect until someone adds player-level detail.
