LLM product recommendations on a WooCommerce store are affordable, but only if the model never sees your whole catalog. It should rerank a shortlist that a dumb, free query already produced. Piero Paialunga builds this exact setup on Towards Data Science with restaurants: a distance filter cuts 10,000 places down to the 50 nearest, then a GPT model picks the best five and attaches an honest fit score to each. It’s the right shape for a store, with a few changes.
Start with the wrong fix, because it’s what most first attempts look like.
Why the obvious version fails
The obvious version is to put your products in the prompt, or to let the model search the catalog for you. Paialunga runs through why that fails: every API call costs tokens, and a model that had to judge every restaurant in a city before answering would burn millions of tokens and take far longer than anyone would wait. A store with a few thousand SKUs hits the same wall. The bill scales with catalog size on every search, which is the one cost you can never get down.
WordPress adds its own failure mode. If the API call runs inside the search request, a PHP worker sits blocked on it, and shared hosting will kill the request long before a slow model with a fat prompt finishes. Then you have to parse free text the model produced, and json_decode failures on live traffic are not fun to debug.
How to build the affordable version
Stage one is a query that costs nothing: WP_Query over published products, keyword search across title and content, a category term if you can pull one out of the phrase. You want around 30 candidates. Stage two sends only those 30 names, prices and short descriptions to the model and asks for a ranked list of product IDs. With structured outputs the answer always matches a schema instead of prose. Stage three is the one most posts skip: cache the ranked IDs in a transient keyed on the normalized query, because the same handful of phrasings covers most search traffic. (I’m not sure transient caching holds up on a very busy store without a persistent object cache, since transients fall back to the options table.)
In PHP that’s roughly:
<?php
function bbioon_recommend( string $query ) {
$key = 'bbioon_rec_' . md5( strtolower( trim( $query ) ) );
$ids = get_transient( $key );
if ( false !== $ids ) {
return $ids;
}
$candidates = new WP_Query( [
'post_type' => 'product',
'post_status' => 'publish',
's' => $query,
'posts_per_page' => 30,
'fields' => 'ids',
] );
if ( count( $candidates->posts ) < 3 ) {
return []; // nothing to rerank, fix stage one instead
}
$ids = bbioon_llm_rerank( $query, $candidates->posts );
set_transient( $key, $ids, 6 * HOUR_IN_SECONDS );
return $ids;
}The rerank stage only sees what stage one kept, so if your keyword search for “cheap vegan dinner” returns two products, the model has nothing to rescue. Paialunga goes for high recall in stage one on purpose: keep the filter loose and do the strict filtering in the prompt. Treat an empty or tiny candidate set as a sign the problem is your prefilter. Both keyword and vector retrieval have blind spots, like exact codes and SKUs. If you’d rather not pay per query at all, a cross-encoder reranker can do stage two locally.
If you want a funnel like this on your own store, the candidate query and the caching are the parts worth tuning to your catalog, and that’s work I do. The model call itself is the easy bit.
Pick your store’s worst performing search phrase today and search it yourself. If it returns fewer than ten products, fix that query before you spend a single token on a model.
