In 14 years of WordPress and backend work, I have watched plenty of developers throw hardware at a bottleneck that was really an indexing or structure problem. Machine learning does the same thing. We hand a model raw coordinates, expect it to figure the rest out, and skip the Fourier Features that would make high-frequency data legible in the first place.
I recently looked at a project that trained a neural network on the Mandelbrot set. It is a brutal test case, because the set is deterministic and has detail all the way down. Feed raw Cartesian coordinates into a standard Multi-Layer Perceptron (MLP) and you get a blurry mess, something close to a low-res JPEG from 1998, and stacking more layers does not help.
The spectral bias bottleneck
The failure has a name: spectral bias. Networks pick up low-frequency components, the broad shapes, first, and they have a hard time with functions that oscillate quickly or carry fine detail. Depth makes it worse, since deeper stacks tend to smooth those variations out.
If you want the WordPress version of the problem, it is querying a huge wp_postmeta table with no index on the meta key. The engine ends up doing work that better structure would have handled up front. The ML fix follows the same logic: transform the input space.
The naive approach that fails
The usual framing is a regression task: map spatial coordinates (x, y) to a smooth escape-time value. Here is the baseline residual MLP that gives you the blurry output:
class MLPRes(nn.Module):
def __init__(self, hidden_dim=256, num_blocks=8, out_dim=1):
super().__init__()
self.in_proj = nn.Linear(2, hidden_dim)
self.blocks = nn.Sequential(*[
ResidualBlock(hidden_dim) for _ in range(num_blocks)
])
self.out_proj = nn.Linear(hidden_dim, out_dim)
def forward(self, x):
x = F.silu(self.in_proj(x))
x = self.blocks(x)
return self.out_proj(x)That network learns the rough blob of the Mandelbrot set and misses every filament and fractal edge. I wrote about a related failure mode in my guide on improving visual anomaly detection models.
Why Fourier features work
The fix, popularized by Tancik et al. (2020), is to run the input through a Fourier Features mapping before it reaches the first linear layer. You project the coordinates onto random directions in a higher-dimensional space using sinusoids.
The mapping behaves like a random Fourier basis expansion, which puts the high-frequency structure right there in the input. The network no longer has to discover those frequencies through a stack of transformations, so even a shallow one can render sharp fractal boundaries.
Implementing multi-scale encoding
One frequency scale usually is not enough for a fractal, so you want a multi-resolution basis. It is the same instinct behind the way I handle complex data structures in the backend, where you break information into indexed scales you can actually query.
class MultiScaleGaussianFourierFeatures(nn.Module):
def __init__(self, in_dim=2, num_feats=512, sigmas=(2.0, 6.0, 10.0)):
super().__init__()
# Split features across frequency bands
k = len(sigmas)
per_scale = num_feats // k
Bs = []
for s in sigmas:
B = torch.randn(in_dim, per_scale) * s
Bs.append(B)
self.register_buffer("B", torch.cat(Bs, dim=1))
def forward(self, x):
proj = (2 * torch.pi) * (x @ self.B)
return torch.cat([torch.sin(proj), torch.cos(proj)], dim=-1)The results: representation vs. architecture
Plug the features into the same MLP and the training curve changes. Without them the model plateaus almost immediately. With them you get coarse-to-fine learning: the general shape first, then the filaments. The win came from a better input, not a bigger model.
If Fourier Features or AI integration in general is eating your dev hours, I can take it off your plate. I have been working with WordPress since the 4.x days, and I can usually tell when a problem wants a bigger server and when it wants a smarter data representation.
Takeaway: the input encoding sets the ceiling
Coordinate-based networks for graphics and an overloaded WooCommerce database run into the same wall: your choice of input encoding decides how much performance you can get. Do not make your logic synthesize complexity that the data layer could state outright. Move the complexity into the input, and the model can spend its capacity finding patterns you already prepared for it.
