We need to talk about AI latency. For some reason, the standard advice for building LLM integrations has become “wait for the full JSON response and then update the UI.” If you’re doing this, you’re killing your performance. I’ve seen sites where the user sits staring at a spinner for 15 seconds because the model is busy “thinking.” In 2026, that’s a legacy mistake. To fix this, you need to implement AI response streaming properly.
When you request 2,000 words from a model, the bottleneck isn’t just the network; it’s the token generation speed. Even a fully optimized AI implementation strategy won’t save you from the physics of inference time. Streaming is the pragmatist’s workaround: instead of waiting for the full payload, you deliver the response token by token as it’s generated.
SSE vs. WebSockets: Don’t Over-Engineer
In the developer community, I often see people reaching for WebSockets the moment they hear the word “real-time.” Unless you are building a complex, multi-agent system where the client and server need a constant, bidirectional conversation, WebSockets are overkill. They introduce unnecessary complexity in state management and server overhead.
For most WordPress-based AI apps, Server-Sent Events (SSE) is the superior choice. It’s a one-way street—server to client—that works over standard HTTP. It’s lightweight, it handles reconnections automatically, and it’s natively supported by the MDN EventSource API. Most importantly, it’s exactly how OpenAI and Claude handle their own streaming endpoints.
The Mistake: The Naive JSON Fetch
Here is what most developers do. They use fetch(), wait for the response to resolve, and then parse the JSON. This is fine for a 100ms API call, but for AI, it’s a UX disaster.
// The "Naive" Approach - Don't do this for AI
async function fetchAIResponse(prompt) {
const response = await fetch('/wp-json/my-ai/v1/generate', {
method: 'POST',
body: JSON.stringify({ prompt })
});
const data = await response.json(); // Site hangs here for 10 seconds
document.getElementById('output').innerText = data.text;
}The Fix: ReadableStream for AI Response Streaming
Specifically, you want to consume the response body as a stream. By using the getReader() method on the response body, you can process chunks of text as they arrive from the model.
// The Senior Dev Approach: Consuming a Stream
async function streamAIResponse(prompt) {
const response = await fetch('/wp-json/my-ai/v1/stream', {
method: 'POST',
body: JSON.stringify({ prompt })
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
let isDone = false;
while (!isDone) {
const { value, done } = await reader.read();
isDone = done;
const chunkValue = decoder.decode(value);
// Append tokens to UI in real-time
document.getElementById('output').innerText += chunkValue;
}
}The Content Validation Gotcha
Furthermore, we need to address the elephant in the room: validation. When you use AI response streaming, you are bypassing your ability to check the content before the user sees it. If the model hallucinates or violates a safety policy at token #400, your UI has already shown the first 399 tokens.
I’ve had cases where a client’s chatbot started giving great advice, only to pivot into a race condition of nonsense halfway through. Consequently, if your app requires strict output guarantees (like generating valid JSON or passing a toxicity filter), you might actually be better off not streaming, or at least running a post-generation validation check that can “undo” the output if it fails.
Look, if this AI response streaming stuff is eating up your dev hours, let me handle it. I’ve been wrestling with WordPress since the 4.x days.
Pragmatic Takeaway
AI response streaming isn’t just about making things “look cool” with a typing effect. It’s a technical necessity for perceived performance. However, don’t just slap stream: true on every request. Evaluate the length of your outputs and your need for pre-rendering validation. If you’re building a chatbot, stream. If you’re generating a structured configuration file, wait. Stability over shine—always.
