Machine learning projects fail for boring reasons, and somewhere along the way the standard advice became throwing more compute at them. Picture the scene: 11:14 PM, a Bayesian optimization sweep grinding through its 200th trial for another 0.2% of AUC. It feels like engineering. It is procrastination, because Machine Learning Problem Framing is hard and tuning hyperparameters is easy.
I thought I had seen every way a tech project can break, then I started auditing “AI-powered” WordPress plugins and custom enterprise builds. Same pattern every time: a well-built model solving the wrong thing. RAND research puts AI project failure at 80%, and the cause is rarely thin data or a weak model. The framing was wrong before anyone wrote a line of code.
The productive procrastination trap
Hyperparameter tuning feels like real work. There is a search space, you iterate, and you get a screenshot of a metric ticking upward that looks great in Slack. If that metric does not map to a decision somebody makes, you are polishing something nobody will use, and the gap between your model’s accuracy and what the business needs keeps widening while you do it.
Zillow’s $500 million loss is the clean example. The model was not what broke; the Machine Learning Problem Framing never got questioned. They framed it as “predict home value” on the assumption of a stable market, when the real problem was operational speed and error asymmetry. Their competitors survived because they tuned the problem rather than the learning rate.
A five-step protocol for machine learning problem framing
Run this before you open a Jupyter notebook or start a training job. It costs a few days of talking to people, which is a lot cheaper than a GPU cluster.
- Name the decision, not the prediction: ask the stakeholder what changes when the model produces an output. If they cannot name a person and an action, pause the project. A model with no decision attached is a report nobody opens.
- Price the two kinds of error: false positives and false negatives never cost the same outside a textbook. On a fraud model, missing fraud might cost $4,000 while a false flag costs $12. Optimize for plain F1 and you are pretending that 23:1 cost ratio does not exist.
- Audit the target variable: is churn a canceled subscription, or a user who quietly stopped logging in? Those need different data hooks and different filters. And if your prediction window leaves the team no time to act, the output arrives dead.
- Simulate the deployment: run a tabletop exercise with 10 synthetic outputs. If the stakeholder will not act on an 85% confidence score, half of what the model does is already wasted.
- Write the anti-target: describe how this project hits every metric and still fails in production. Inverting it surfaces the operational bottlenecks you have been stepping around.
I have written before about how your environment dictates ML success, and environment covers more than your Docker stack. It includes the conceptual environment the model has to live in.
Is it a tuning problem or a framing problem?
When performance stalls, reaching for Optuna is the reflex. Check the framing first. If nobody has confirmed that the target maps to a real decision, or if the training signal carries shortcuts (the infamous model that learned to spot rulers instead of cancer), grid search will not rescue it. Garbage signal survives any amount of searching.
If machine learning problem framing is eating your dev hours, hand it over to me. I have been wrestling with WordPress and high-level logic since the 4.x days.
What seniority actually buys you
Junior devs get paid to tune and deploy. Senior devs get paid to frame. The bottleneck in this work is not compute, it is the conversation between the person building the model and the person who has to act on its output. Spend next week on the problem instead of the learning rate.
On bridging that gap, I also went through ML lessons for WordPress development.
