The default advice for RAG and AI agents right now is to stuff the context window and let the model sort it out. That advice is costing you performance. After 14 years of untangling complex WordPress architectures, the pattern looks familiar: it is the same mistake as installing 50 plugins to cover one feature. Coverage goes up, and so does the mess. What you are left with is a maintenance problem and a bottleneck.
A framework called the Bits-over-Random Metric puts a number on something I kept running into in production LLM systems. Most developers open their retrieval dashboard, see a high Success@K, and call it a win. What that number hides is context pollution. The model finding the “needle” tells you nothing about whether it can use the needle after you dragged half the haystack in with it.
The selectivity paradox: when 99% success tells you nothing
Classic information retrieval rewards high recall. With agents, the context window is a fixed budget and every chunk in it has to earn its place. The 99% Success Paradox research shows that at high retrieval depths (K), your “success” starts to look like random chance. The Bits-over-Random Metric is that comparison: how much better is your retrieval than picking items at random?
Show the model all 10 of your 10 tools and recall is 100% with selectivity of zero. That is not routing. You have handed the reasoning back to a model that now has to read ten tool descriptions to answer one question. This is where the “Collapse Regime” starts, the point where more candidate tools raise apparent coverage while the agent gets worse at picking one.
Fixing the collapse with staged tool retrieval
When I build WordPress AI Agents, I never expose the whole tool list. The prompt gets a staged shortlist instead. Route to a domain first, something like “WooCommerce” or “System Config”, then pull a small set of tools from inside that domain. Your Bits-over-Random Metric stays high and the token bill stays low.
A stripped-down version of that routing looks like this, so the model never sees a pile of plausible but wrong options:
<?php
/**
* bbioon_staged_tool_retrieval
* Prevents context pollution by filtering tools based on intent.
*/
function bbioon_get_scoped_tools( $user_intent ) {
$all_tools = [
'order_status_update' => [ 'domain' => 'ecommerce', 'desc' => '...' ],
'inventory_sync' => [ 'domain' => 'ecommerce', 'desc' => '...' ],
'user_password_reset' => [ 'domain' => 'security', 'desc' => '...' ],
'firewall_toggle' => [ 'domain' => 'security', 'desc' => '...' ],
];
// Naive approach: return all tools (Low BoR)
// Refactored approach: filter by intent domain (High BoR)
$domain_map = [
'check my sales' => 'ecommerce',
'secure my site' => 'security',
];
$target_domain = $domain_map[ $user_intent ] ?? 'general';
return array_filter( $all_tools, function($tool) use ($target_domain) {
return $tool['domain'] === $target_domain;
});
}Why context purity beats coverage
A lot of software engineering work is turning into context engineering. Every duplicated chunk and every loosely related example you feed the prompt behaves like noise in a race condition, competing for the model’s attention. So the question is no longer “how large must K be?” It is “how small can my shortlist get before performance drops?”
Dozens of personal AI agent development projects I have looked at failed for the same reason: they mistook coverage for skill. If your routed shortlist scores the same as handing the model every tool, the routing layer is doing nothing. If adding tools makes the score drop, you are already in the collapse zone.
If this Bits-over-Random Metric work is eating your dev hours, hand it over. I have been building on WordPress since the 4.x days and I know how to keep an architecture clean.
What to measure instead
Retrieval used to be about finding the needle in the haystack. With an LLM on the other end, the job is to bring the needle without half the haystack attached to it. Track the Bits-over-Random Metric and you can tell an agent that is choosing apart from one that got lucky because you handed it everything. Keep the shortlist small.
