We need to talk about RAG. The standard advice for building a search or comparison tool has settled into “chunk it, embed it, and hope for the best.” For a chatbot on a recipe blog, that is fine. For 200-page credit agreements or clinical research papers, the naive “Chunk-Embed-Match” approach will eventually hand someone a confident wrong answer. Reliable comparison needs Structure-Aware Document Intelligence.
The problem with flat RAG
I have watched this play out with clients who want to compare two legal contracts. They use a standard vector database. The system retrieves “Section 4.1” from Document A, but since it arrives as a bare chunk, the LLM never learns that Section 4.1 is an exception to a rule defined ten pages earlier. The comparison comes back semantically shallow, and often just wrong.
In complex documents, meaning sits in the hierarchy rather than in isolated fragments. A retrieval pipeline that ignores headers, breadcrumbs and section relationships is doing keyword matching on text where a wrong match is expensive. That is why flat RAG fails on complex enterprise documents.
How the Proxy-Pointer framework works
Proxy-Pointer builds the document’s structure into retrieval itself, which is what Structure-Aware Document Intelligence means in practice. Instead of indexing raw text, it indexes structural pointers, or breadcrumbs: Document > Article 5 > Section 5.2 > Subsection (a).
This architecture uses a two-stage retrieval process:
- Stage 1: Find the relevant sections in the primary document using hierarchical breadcrumbs.
- Stage 2: Use the context of those sections to run a targeted cross-retrieval in the second document.
Upstream extraction matters here too. Tools like LlamaParse turn messy PDFs into clean Markdown that keeps the hierarchy intact, so the LLM always knows where it is in the document.
Implementing Structure-Aware Document Intelligence
If you are building this in Python, your indexing logic should read like a tree traversal instead of a text dump. Here is roughly how the section comparison logic looks:
import json
def bbioon_build_structural_index(markdown_content):
# Logic to parse headers and create a JSON map of the document tree
# This prevents the LLM from losing track of parent-child relationships
structure_map = {
"root": "Loan Agreement",
"nodes": [
{"id": "sec_1", "header": "Article I: Definitions", "level": 1},
{"id": "sec_1.1", "header": "Section 1.1: Collateral", "parent": "sec_1", "level": 2}
]
}
return structure_map
def bbioon_compare_sections(doc1_context, doc2_index, criteria):
# Stage 2: Cross-retrieval
# Instead of a global search, we search Doc 2 within the
# semantically aligned neighborhood of Doc 1's context.
query = f"Find analogous sections for {criteria} given this context: {doc1_context}"
# Target search in Doc 2's vector space
passThe same logic works inside a CMS. I have put it into custom WordPress document portals for law firms, using a headless setup where the WP REST API feeds the document structure to a Python backend and the results render into a React comparison dashboard. My write-up on Proxy-Pointer RAG: why structure beats plain chunking goes through the accuracy benchmarks.
Why this matters in production
Ship an AI tool to a legal professional and their tolerance for hallucinations is zero. A “vibe check” is not an evaluation strategy. With Structure-Aware Document Intelligence, an answer is anchored to the document’s real layout rather than to whatever the embedding happened to surface.
For the full implementation, start with the Proxy-Pointer GitHub repo. It includes the criteria_validator and section_selector logic that does the heavy lifting.
If this Structure-Aware Document Intelligence work is eating your dev hours, I can take it on. I’ve been working with WordPress since the 4.x days.
The takeaway
A long document has a structure, and your retrieval pipeline should use it. Whether you are comparing credit agreements or research papers, a pipeline that knows where a clause sits can tell a main clause from an exception to it, which flat chunking cannot do.
