The support agent in your plugin works fine for three turns. On the fourth it dies with a context length exceeded error, and the order state it had pieced together goes with it. That is an AI agent context management failure, and it’s the predictable end of a design where the chat transcript is the only record of what happened.
Hussen Mohammed Ibrahim hit the same wall building a scientific agent on local models, and his writeup of the infrastructure behind it is worth reading even if you’ll never touch a GPU. His agent resends a 36,000-token system prompt plus tool schemas on every loop iteration, and one analysis makes 50 to 80 tool calls. No WordPress host is running that. But two of his findings decide whether a cloud-backed agent in a plugin feels fast and survives a long run, and neither one needs a GPU.
What each iteration sends
An agent loop isn’t one call. The model picks a tool, you run it, append the result, and send the whole conversation back. By iteration 40 the model is reading a transcript of everything it did plus the same fixed instructions it already read 39 times. On his hardware that cost 10 to 15 seconds per iteration before tuning, and 1 to 3 seconds after. On a cloud API the same waste shows up as latency and a bigger per-call bill instead of a slow GPU, so the loop shape matters to you either way.
Summarizing when the window fills is the wrong fix
The common answer is compaction: when the window fills, ask the model to summarize the history and continue from the summary. For a chatbot that’s fine. For anything that depends on exact values it loses the wrong things. His example is scientific, quality control thresholds and clustering resolution. The store version is which SKUs matched the filter, which price rule fired, which order IDs were already handled. A summary saying the agent checked the pending orders discards all of that, and the agent then answers questions about its own work using details it no longer has.
The trimming math goes wrong in a quieter way. If your threshold is half the context window but you never subtract the system prompt, the tool schemas, and a reserve for the completion itself, you’re trimming against a budget that was overspent before the first history message arrived.
Why your system prompt is breaking the cache
Every serious serving stack caches prompts by shared prefix. The vLLM prefix caching docs describe it plainly: compute the shared part once, reuse it for any request that starts the same way. In the article, time to first token on the real 36,000-token prompt dropped from 11,470ms cold to 706ms warm on an A100. Anthropic and OpenAI run the same idea server-side, and Anthropic’s context engineering post covers what that means for agent design.
Plugin authors usually break this by making the system prompt dynamic: cart contents in the prompt, today’s stock levels, the current order list. Change one token near the top and nothing after it matches the cache, so every call is cold again. Adding a tool mid-session does the same thing. Keep the system prompt and tool list static for the life of the run, and put volatile state in the last user message instead.
That ordering rule costs nothing, and on a loop it’s the biggest saving I know of.
Keeping state outside the transcript
The part I’d copy wholesale is his world state idea, which means you stop treating the transcript as the record. Keep a run log outside the conversation, one entry per completed tool call with the exact arguments and the exact outcome, and render the recent entries back into the prompt on each iteration. Then trim the transcript aggressively, because the ground truth lives somewhere that doesn’t get trimmed. In his build the rendered state stays under 1,000 tokens and carries exact parameters a prose summary would have dropped.
function bbioon_log_agent_step( int $run_id, string $tool, array $args, string $outcome ): void {
global $wpdb;
$wpdb->insert(
$wpdb->prefix . 'agent_run_log',
[
'run_id' => $run_id,
'tool' => $tool,
'args' => wp_json_encode( $args ),
'outcome' => $outcome,
'created' => current_time( 'mysql' ),
],
[ '%d', '%s', '%s', '%s', '%s' ]
);
}In a plugin that’s a custom table created with dbDelta, or a custom post type if the client wants to browse runs in wp-admin. If you want the background, I wrote earlier about how agent architecture in WordPress is mostly plumbing, and about how the fixed prefix drives your AI agent token costs.
The trim rule comes straight from the article: available history equals the context limit minus system tokens minus tool schema tokens minus a completion reserve minus a safety margin. On a 262,000-token window that leaves room for a long run. On a 32,000-token window it tells you up front that a long session can’t work, which beats finding out on iteration three. User messages never get trimmed; they carry the intent.
If you’re wiring an agent into a store and want the run log, the budget math and the prompt ordering done properly, that’s work I take on. Tell me what you’re extending and I’ll point at where it breaks first.
When not to bother
If your AI feature is one call, generate a product description or summarize a review, skip all of it. No loop means no growing prefix and no record to keep. Before changing anything on an existing loop, log time to first token for every call in one real session. If the second call is as slow as the first, your prefix is churning, and the fix is a static system prompt. I’ve assumed one agent owning one context here, and I’m not certain the same trimming rules hold once subagents each carry their own window.
