Most data systems don’t collapse overnight. They rot one “quick fix” query at a time, until nobody can say where a number came from. The advice going around in the WordPress world and in dev circles generally is to move the data first and sort out the logic later, which is how you end up with a warehouse full of queries nobody wants to touch. What is missing is a disciplined SQL Data Transformation layer, and that is what turns a clean warehouse into the SQL jungle.
How ETL became the ELT jungle
ETL came first: extract, transform, load. You cleaned the data before it hit the warehouse, which was rigid, heavy, and a real pain for a team that wanted to ship this week. ELT flipped the last two steps, so you extract, load, and then transform in place. Analysts got a lot more power out of that, and we quietly gave up the engineering discipline that had kept the logic sane.
When you democratize transformations without structure, business logic starts to scatter like legacy code in a 10-year-old plugin. One team calculates “Active Users” in a Tableau dashboard; another does it in a scheduled cron job. Before long, you have three different definitions for the same metric, and nobody knows which one to trust. If you’ve ever dealt with robust historical data analysis, you know exactly how fast this scales into a nightmare.
What a real SQL transformation layer needs
Getting out of the jungle means treating transformations like software instead of scripts you keep in a folder. One model, one SQL file, one responsibility. That 500-line query joining twenty tables becomes a handful of small files you can read in a sitting.
-- The "Bad" Approach: The Mega-Query
SELECT
o.id,
c.name,
SUM(o.total) as revenue,
-- 400 more lines of nested subqueries and CASE statements
FROM orders o
JOIN customers c ON o.customer_id = c.id
GROUP BY 1, 2;
-- The "Senior" Approach: Modular Models (e.g., dbt style)
-- models/staging/stg_orders.sql
SELECT id, customer_id, total, created_at FROM raw.orders;
-- models/marts/fct_revenue.sql
SELECT
customer_id,
SUM(total) as daily_revenue
FROM {{ ref('stg_orders') }}
GROUP BY 1;Splitting things up is also what gives you lineage. You can see what will break before you run a DROP TABLE, not after. dbt and SQLMesh get dismissed as shiny tooling, but what they actually do is force the discipline. Tests for nulls and uniqueness run as part of the development cycle, so a bad assumption shows up in your terminal instead of in somebody’s dashboard.
Anti-patterns worth refactoring now
- If your revenue calculation lives inside a PowerBI measure, you have already lost the argument about a single source of truth. Push the logic down into the database so every tool reads the same number.
- A query that needs a 15-minute walkthrough before anyone dares change it is a bottleneck with a person attached to it. Break it into intermediate staging tables.
- Editing a view directly in the production warehouse is live-coding on a busy WooCommerce site. It works right up until two people do it in the same hour. Everything belongs in version control.
I have watched teams stall because they assumed 90% accuracy in SQL is enough. Production does not grade on a curve. Without a structured SQL Data Transformation framework underneath it, every number you hand to a stakeholder is a guess.
If this kind of work is eating your dev hours, I do it for a living. I have been wrestling with WordPress and awkward database architecture since the 4.x days.
From jungle to infrastructure
ELT was supposed to make us faster, and it did, right up to the point where nobody could explain a number in a dashboard. Speed without structure is just debt with a shorter fuse. Put version control, tests, and modular models into the transformation layer and the jungle turns back into infrastructure. Write software, not scripts. Ship it.
