The default advice for how to detect LLM hallucinations is to hand the output to another language model and ask whether it looks right. That is asking a colorblind person to sort paint samples. It is circular, and it costs you a second inference call on every response. A system that hallucinates cannot police itself, which matters as soon as you ship AI features into WordPress for real users.
Years of debugging race conditions and legacy PHP hooks taught me to read error logs. AI breaks in a way that never reaches the log at all. What I want instead is a deterministic check, and the vector space these models already work in turns out to be a good place to look for one.
The geometry of a lie
Feed text into an embedding model and you get a vector, a point in high-dimensional space, with semantically similar texts landing near each other. Distance is not the only structure in there. Connect a question vector to its answer vector and you have a displacement vector.
In a grounded system those displacement vectors point in consistent directions inside a given domain. Ask five legal questions and the vectors joining them to their answers should come out roughly parallel. A hallucinated answer can read perfectly well and still send its displacement off in some unrelated direction. It is the red bird flying against the flock.
How to detect LLM hallucinations with displacement consistency
Call the method Displacement Consistency, or DC. It needs no source document and no second LLM at inference time. You measure how closely a new displacement matches a reference set and score it on that. The same habit of trading validation for speed came up in what I wrote about technical debt in AI development.
You need a small calibration set of grounded pairs to start. For a new query, find its neighbors in the embedding space and average their displacement directions. An answer whose displacement drifts far from that average is probably fabricated. The Python below is the stripped-down version of that logic, the sort of thing you would run as a microservice alongside your WordPress site:
import numpy as np
from sklearn.metrics.pairwise import cosine_similarity
def bbioon_calculate_dc_score(q_vec, a_vec, neighbor_qs, neighbor_as):
# Calculate the displacement of the current pair
v_current = a_vec - q_vec
# Calculate displacements for grounded neighbors
v_neighbors = neighbor_as - neighbor_qs
# Get the mean direction of the flock
v_mean_direction = np.mean(v_neighbors, axis=0)
# The DC Score is the cosine similarity (0 to 1)
# Higher = Grounded, Lower = Hallucination
score = cosine_similarity(
v_current.reshape(1, -1),
v_mean_direction.reshape(1, -1)
)
return float(score[0][0])The catch: domain locality
DC scores well on benchmarks, close to perfect discrimination on some of them, but it is not a general truth detector. Grounding here is a domain-local geometric property. The arXiv paper on this makes the point that the grounded direction for legal advice is not the grounded direction for medical advice, so the reference set has to be calibrated per use case.
If hallucination detection is eating your dev hours, I can take it off your plate. I have been working with WordPress since the 4.x days.
What this means for WordPress developers
A judge LLM carries the same biases as the generator that produced the text, so it waves through the same mistakes. The structure already sitting in your embeddings is cheaper and steadier. Take the cosine similarity between displacement vectors and your integration gets a usable signal for when it is guessing.
