Search in the WordPress world has settled on one piece of advice: throw your data into a vector database and call it done. That advice is what wrecks your accuracy. Anyone who has shipped an LLM feature has hit the same wall, and getting from “okay” results to something you would put in front of a customer takes advanced RAG retrieval.
I have spent 14 years wrestling with complex data structures in WooCommerce, and the same mistake turns up with every new wave of technology. We reach for bi-encoders, the models that turn text into embeddings, because they are fast. They are also limited: they squeeze all the meaning into a single vector before they have seen the query. Nuance and outright contradictions get flattened, and your users get results that have nothing to do with what they asked for.
Where bi-encoders run out of road
Most semantic search systems use a bi-encoder. It scales because you precompute embeddings for the whole document library, so the lookup itself is cheap. Someone searches for “cheap hotels in Tokyo” and the system returns the vectors sitting mathematically closest to that phrase. It will happily put “Luxury hotels” on top because “hotels” and “Tokyo” both match, while “cheap” quietly drops out of the equation.
That gap is what advanced RAG retrieval closes. Instead of one pass, you run two stages: the bi-encoder casts a wide net for recall, then a cross-encoder re-scores what came back for precision. A cross-encoder reads the query and the document together, with full self-attention across every token.
Building a two-stage reranking pipeline
You cannot run a heavy transformer model on every query in production WordPress. That bottleneck arrives faster than a badly written WP_Query against a million rows, so treat retrieval as a funnel instead. I covered the architectural side of this in an earlier post on treating AI memory properly.
- Stage 1: Fast retrieval from a vector DB such as Pinecone, or from a local BM25 index. Pull the top 50 candidates.
- Stage 2: Reranking with a cross-encoder model. The query and those 50 candidates go to a reranking API.
Here is roughly how that looks in a custom WordPress plugin talking to an external reranking service like Cohere Rerank:
<?php
/**
* Example of integrating a reranking stage into a WP search pipeline.
* Prefixing with bbioon_ as per senior dev standards.
*/
function bbioon_advanced_rag_rerank( $query, $initial_results ) {
$api_key = 'YOUR_COHERE_API_KEY';
$documents = array_column( $initial_results, 'content' );
$response = wp_remote_post( 'https://api.cohere.ai/v1/rerank', [
'headers' => [
'Authorization' => 'Bearer ' . $api_key,
'Content-Type' => 'application/json',
],
'body' => json_encode([
'model' => 'rerank-english-v3.0',
'query' => $query,
'documents' => $documents,
'top_n' => 5,
]),
]);
if ( is_wp_error( $response ) ) {
return $initial_results; // Fallback to raw retrieval if API fails
}
$data = json_decode( wp_remote_retrieve_body( $response ), true );
// Sort and return the reranked documents
return $data['results'];
}What cross-attention actually buys you
Cross-attention is the reason the second pass earns its keep. In a bi-encoder the query and the document never talk to each other until the final comparison. A cross-encoder concatenates them, so every word in the query can attend to every word in the document at every layer of the model. That is how it catches a contradiction like “no sugar” against “contains sugar,” which plain embedding search usually misses. It is the same idea behind the way we handle RAG conflicts: make the model look harder at the context.
Keeping latency down under traffic (ColBERT and caching)
On a busy WooCommerce store, 200ms added to every search is a risky trade. For those sites I look at ColBERT late interaction, or at semantic query caching. Use a cross-encoder to spot duplicate queries at the cache layer and 70-80% of your traffic never reaches the reranker at all. A model like MiniLM-L6 also lands in a good spot between speed and accuracy.
If this retrieval work is eating your dev hours, hand it over. I have been wrestling with WordPress since the 4.x days.
What to change first
Do not trust your embedding model blindly. When a RAG pipeline returns “okay” answers, the fix is rarely a bigger LLM; it is a better retrieval strategy. Add reranking, try a couple of cross-encoders, and keep an eye on the latency budget while you do it. Finding the closest vector is different from working out what the user meant inside your particular domain.
