The standard RAG advice in the WordPress and enterprise world has settled on “just chunk your documents and throw them in a vector database.” It is slow, and it produces answers that are confident and wrong. After 14 years of debugging broken site architectures, my read is that math-based similarity search, plain vector RAG, runs out of room as soon as a question needs real structured reasoning.
The newer answer is “vectorless RAG,” or PageIndex-style retrieval. It gets close to 99% on financial benchmarks, which is remarkable, and it comes with a catch: it is slow, and the LLM API fees will drain your budget before anything reaches production. Proxy-Pointer RAG is the middle path.
The scaling wall of vectorless RAG
Vectorless systems like PageIndex build a hierarchical tree of summaries. Rather than doing math, an LLM reads the table of contents and picks the right section. Retrieval quality is genuinely good; the indexing cost is what kills it. A 130-page report can take 130 or more LLM calls just to index. Multiply that by an enterprise knowledge base of 500 documents and you have spent thousands of dollars in tokens before the first user question arrives.
Latency is the second problem. Making a user wait while an LLM walks the tree, before synthesis has even started, is not something a production interface can absorb. If a query takes more than 3 seconds to start streaming, you have already lost that user.
How Proxy-Pointer RAG works
Proxy-Pointer RAG starts from the observation that structural awareness does not have to come from an expensive LLM. You can encode the structure into the embeddings and the metadata pointers instead. Three engineering techniques get you to roughly 90% of a reasoning-based retriever’s accuracy at the cost of an ordinary vector search.
1. Skeleton trees, indexed for free
Do not spend LLM calls summarizing every node during ingestion. Build a skeleton tree with plain regex. Structured documents, PDFs and Markdown alike, almost always carry a clear heading hierarchy, and parsing those headings into a nested JSON tree takes milliseconds. You end up with a usable table of contents without paying for a single LLM call at index time.
2. Breadcrumb injection
One reason vector similarity fails is that each chunk is an island with no idea where it came from. The fix is to prepend the full ancestry path from the skeleton tree, the breadcrumbs, to every chunk before it reaches the embedder.
So instead of embedding “Revenue grew by 5%,” you embed “[Report > Chapter 1 > Financial Performance] Revenue grew by 5%.” FAISS, or whichever vector store you use, then has the structural location of the text baked into what it matches at query time.
3. Metadata pointers instead of chunks
In standard RAG the retrieved chunk is the context. Here the chunk is only a proxy. Each one carries a pointer, a pair of line boundaries, back to its full section in the source document. When a chunk matches, you follow the pointer and hand the synthesis LLM the entire contiguous section. That is what removes the split-sentence and lost-context hallucinations naive chunking keeps producing.
<?php
/**
* Simple Logic for Proxy-Pointer Breadcrumb Injection
* Prepend hierarchy to chunks before sending to the vector database.
*/
function bbioon_prepare_proxy_pointer_chunks( array $hierarchy_node, string $parent_crumb = '' ) {
$current_crumb = $parent_crumb ? $parent_crumb . ' > ' . $hierarchy_node['title'] : $hierarchy_node['title'];
$chunks = [];
// Split text only within this section's boundaries
$section_content = bbioon_get_content_by_lines( $hierarchy_node['start_line'], $hierarchy_node['end_line'] );
// Inject breadcrumb as structural context
$enriched_text = "[{$current_crumb}]\n" . $section_content;
// Store with pointers back to the original source
$chunks[] = [
'text' => $enriched_text,
'metadata' => [
'node_id' => $hierarchy_node['id'],
'start_line' => $hierarchy_node['start_line'],
'end_line' => $hierarchy_node['end_line']
]
];
return $chunks;
}If this kind of work is eating your dev hours, I can take it on. I have been working with WordPress since the 4.x days, and I have built enough AI-backed custom solutions to know where the slow parts hide.
The takeaway: fix the index, not the model
If you are seeing failure in your agentic RAG, a bigger and more expensive model is rarely the fix. A better index usually is. Skeleton trees and metadata pointers close most of the distance between cheap but dumb retrieval and accurate but expensive retrieval, and they do it by changing what you embed rather than how many tokens you burn.
