For years the standard advice on any AI project was simple: vacuum up every scrap of data you can find, dump it into one centralized bucket (usually S3 or a big SQL cluster), and start training. That approach is hitting a wall, and server cost is the smallest part of it. Much of the data worth training on sits behind silos we cannot open and should not want to, while privacy rules and latency make moving the rest of it expensive.
Federated Learning goes the other way. Instead of moving the data to the model, you move the model to the data. After a decade of fighting database bottlenecks and race conditions, I read keeping data local as an architectural requirement for edge computing rather than only a privacy win.
Why centralized machine learning is failing
A traditional setup assumes a stable connection and enough bandwidth to shift petabytes around. Hospitals, mobile phones and autonomous cars do not work that way: their data is unstructured, sensitive, or simply too heavy to move. In healthcare, up to 97% of data goes unused because regulations like GDPR and HIPAA keep it trapped in silos.
Centralizing that is a legal problem before it is a technical one. It also hands you a single point of failure and one very attractive target, because if the central bucket falls, everything in it falls with it. Federated Learning inverts the whole arrangement.
Moving the model instead of the data
Federated Learning is a collaborative training setup. Raw data stays on the device, the “client,” and only the learned weights or gradients travel back to a central server. Google Gboard already works like this for next-word prediction, which is how it keeps improving without reading your private texts in the cloud.
For how this plays out in one industry, I wrote up Winning at Federated Learning Credit Scoring, where privacy runs straight into financial fairness.
Horizontal vs. vertical federated learning
- Horizontal federated learning means clients share a feature space but hold different samples. Two WooCommerce stores on the same database schema with different customers, for instance.
- Vertical federated learning means clients hold the same samples with different features. A bank and an online store both have data on the same person, and each sees a different slice of their behavior.
How the federated loop actually runs
Training happens in rounds rather than one pass. Each round, the server picks a subset of clients and ships them the current global model. Every client trains locally, usually with stochastic gradient descent, and sends back only the updates. The server merges those updates, typically with an algorithm called FedAvg, into a new global model.
Here is Federated Averaging in a few lines of NumPy. The server combines the client weights in proportion to how much local data each client trained on.
import numpy as np
# Client models after local training (updated weights)
client_weights = [
np.array([1.0, 0.8, 0.5]), # client 1
np.array([1.2, 0.9, 0.6]), # client 2
np.array([0.9, 0.7, 0.4]), # client 3
]
# Number of samples at each client
client_sizes = [100, 200, 700]
# Total samples across all selected clients
total_samples = sum(client_sizes)
# Initialize global model
global_weights = np.zeros_like(client_weights[0])
# FedAvg: Weighted average of local models
for weights, size in zip(client_weights, client_sizes):
global_weights += (size / total_samples) * weights
print("Aggregated Global Model:", global_weights)The third client holds the most data at 700 samples, so it pulls the global model further than the other two. The result leans toward the majority while the smaller datasets still move it.
Non-IID data and other bottlenecks
I would be selling you something if I said this part was easy. The first gotcha is Non-IID data (Independent and Identically Distributed). Every client holds different data, so local models can drift in different directions and leave the global model unstable. The second is the network. Train across 1,000 mobile devices and 200 of them may drop out mid-round on a weak 5G signal.
For building these systems, look at the Flower framework. It handles the client-server communication so you are not writing that layer from scratch.
If Federated Learning is eating your dev hours, I can take it on. I have been working on WordPress and awkward backend integrations since the 4.x days.
Where this goes next
Federated Learning changes who holds the data during training, and that is the part worth caring about. Moving the model to the data is what makes the trade-off between privacy and utility workable. The next post in this series goes through the Flower framework and how to run this in production. Until then, be choosier about what you centralize; your legal team will notice.
