On a WooCommerce store, a bad search result costs money in a way a bad docs search doesn’t. The customer who searches “wireless headphones under 50” and gets phone cases either retries or leaves, and you never find out which one happened. So when a store’s new semantic search starts missing, the reflex I keep running into is to bolt a cross-encoder reranker on top of the embeddings. It sounds like the middle rung of a ladder: smarter than cosine, cheaper than the LLM.
Angela Shi tested that layer properly on Towards Data Science. She ran the same failure cases through four embedding models and three off-the-shelf rerankers, and the ladder mostly isn’t there. On four of the five query shapes where a cross-encoder was supposed to win, it matched the embedding alone or did worse. My read for store work: the reranker is a narrow tool, and most of what it would fix on a WooCommerce catalog is cheaper to fix with data you already have.
Does a cross-encoder reranker fix weak retrieval?
Mechanically the two stages score differently. A bi-encoder turns the query and the document into vectors separately and compares them with cosine, so all documents can be embedded ahead of time. The cross-encoder reads the query and the passage together in one forward pass and returns a relevance score, so every query-document pair gets scored at query time. That joint attention is why people expect it to catch what embeddings miss, and the sentence-transformers docs recommend exactly that pairing: retrieve top-100 with the bi-encoder, rerank with the cross-encoder.
The test results undercut the ladder. On a “who signed the contract” shape, where the corpus holds one passage about how signing works and one actual signature line, the only scorer out of seven that put the signature first was all-MiniLM-L6-v2, a free embedding from 2021 with 22M parameters. The literal-token trap, where the query “hot dog” pulls in “the dog basked in the hot sun”, was fixed only by text-embedding-3-large, with no reranker involved. On a synonym test with a lexical distractor, bge-reranker-large and ms-marco-MiniLM-L-12-v2 both fell for the trap that smaller models avoided. Bigger didn’t mean more accurate here; sometimes it meant more lexical bias.
text-embedding-3-large alone often matched or beat ada-002 stacked with bge-reranker-base (278M parameters). If the choice is between upgrading the embedding and adding a reranker, these tests point at the embedding.
When reranking is worth the latency
A cross-encoder can’t be precomputed, because the query is part of its input. In the article’s setup, wiring bge-reranker-base over the top-100 pushed latency into the hundreds of milliseconds before the LLM even started. Store search is something customers poke repeatedly, so that cost lands on every submit. If the answer layer is a chat interface on top, the wait stacks up.
The clean win in the whole grid was signal dilution: the same answer sentence presented alone versus buried inside a 70-word paragraph. The rerankers put the short answer first and pulled the buried one to second, where the cheaper embeddings had it third or worse. On a store, that’s shipping and returns policy pages, long variable-product descriptions, care instructions. If your customers ask questions whose answers sit inside long blocks of text, that’s the one shape where the layer pays. text-embedding-3-large got there at the embedding stage too, so you can fix this at either stage.
Also check the pool size you’d hand it. The article’s position, which matches what I see on catalogs, is that a reranker’s value is proportional to the candidate pool it inherits. If search already narrows a catalog of a few thousand products to 80 through category and attribute filters, you’re re-ordering a set the customer would scroll anyway. WooCommerce gives you the metadata for that narrowing almost for free: product_cat, the pa_* attribute taxonomies, price and stock filters, all usable in a tax_query before any vector math runs.
What to fix before adding one
Three things, none of which need a GPU. SKUs and model numbers belong in an exact-match lookup. The article found the scorers disambiguate ZRX-2025-A from ZRX-2024-B fine in a tiny test, but at catalog scale the identifier becomes a sliver of the cosine signal. Synonyms: the article’s curated mapping from user vocabulary to document vocabulary is, on a store, your attribute terms plus a small synonym list. A shop selling “notebooks” that customers call “laptops” fixes more misses with one synonym row than with a 278M-parameter model. And listing questions, “show me all your wool coats”, ask for a complete set, which every ranked pipeline will silently cut to top-k.
Catching that last shape is a few lines of PHP:
function bbioon_route_product_question( string $query ): string {
// "all", "every", "list" mean the user wants the complete set.
if ( preg_match( '/\b(all|every|list)\b/i', $query ) ) {
return 'query';
}
return 'vector';
}Then the vector search handles what’s left, the fuzzy descriptive query, which is what embeddings are good at. This lines up with what I wrote about stopping at embeddings for WooCommerce search, and if your pipeline runs in PHP, the per-query cost of an extra model matters more than most estimates assume, as I covered in RAG cost control on PHP hosting. If you want a second pair of eyes on whether your store’s misses are reranker problems or metadata problems, that’s work I do; send me ten failing queries and I’ll tell you which pile they fall in.
One thing you can do today is pull your last 50 customer searches from wherever they’re logged (the access logs will carry the s and post_type=product parameters if nothing else is) and sort them into three piles: SKU or model lookups, listing questions, and everything descriptive. Whichever pile is biggest is your next build, and only the third pile is a reranker’s business.
