We teach machine learning badly. The standard advice for anyone adding “AI features” to an app is to import a library and call .fit(), which leaves you helpless the first time the predictions go strange. Treat a Linear Regression Projection as a black box and you will never debug it. Read the math instead and it turns out to be high school geometry with bigger numbers.
I thought I had seen every way a prediction model can break, and then a client asked me to add house price forecasting to their custom WooCommerce backend. Their residuals were wild and nobody could say why. The data was fine. What they had missed is that linear regression is a projection problem.
Vectors are more than arrays
In PHP or JavaScript a vector is an array of numbers. In geometry, (2, 4) is a set of instructions: two units right, four units up. Connect the origin to that point and you have magnitude and direction. The magnitude is the square root of the sum of the squares, which is the Pythagorean theorem you already met at school.
// The manual way to calculate magnitude in PHP
function bbioon_get_magnitude(array $vector) {
$sum = array_sum(array_map(fn($v) => $v ** 2, $vector));
return sqrt($sum);
}
$v = [2, 4];
echo bbioon_get_magnitude($v); // ~4.47The dot product measures agreement
Given two vectors, say your actual data (y) and your feature (X), you want to know how much they agree. That is what the dot product tells you. A positive result means they lean the same way. Zero means they are orthogonal, so there is no correlation at all. A Linear Regression Projection is the search for the point where the error vector turns orthogonal to the feature space.
The forest analogy
Your house sits at (2, 4), deep in a forest. A highway runs in the direction of (6, 2). Rain has made the mud road to your house unusable, so you drive along the highway, park, and walk the rest of the way. Where do you park to keep the walk as short as possible?
You park where your walking path meets the highway at 90 degrees. That point is the projection of your home vector onto the highway vector, and it is what scikit-learn computes when it minimizes the sum of squared errors.
Calculating the linear regression projection
Finding that parking spot takes a scaling factor. Take the dot product of the two vectors and divide it by the squared length of the highway. Here that is 20 / 40, so 0.5. Scale the highway vector (6, 2) by 0.5 and the parking spot is (3, 1).
Here is the same job in scikit-learn, except now you know what the red line is actually doing:
import numpy as np
from sklearn.linear_model import LinearRegression
# The "Highway" (Size) and the "Home" (Price)
X = np.array([1, 2, 3]).reshape(-1, 1)
y = np.array([11, 12, 19])
model = LinearRegression()
model.fit(X, y)
# These results are just the algebraic manifestation of our projection
print(f"Slope (Beta 1): {model.coef_[0]}")
print(f"Predictions: {model.predict(X)}")If you want to see these patterns in bigger systems, I wrote up 3 Machine Learning Lessons for WordPress Development and some Senior Dev Insights on Applied Statistics.
If Linear Regression Projection work is eating your dev hours, hand it to me. I have been wrestling with WordPress since the 4.x days.
The takeaway
Every line of best fit is a vector projected onto a span of features. Once the geometry is visible, the model stops being magic and you can reason about what it will do to your data. Part 2 moves from the intuition to the exact matrix implementation.
