Angela Shi’s minimal RAG pipeline is about a hundred lines of Python with no vector database, no framework, and no agents. It parses a real 15-page PDF, answers a question, and highlights the exact source lines on the page. What stayed with me is the answer schema, because that’s the one piece you can drop straight into WordPress work.
What the minimal RAG pipeline does
The pipeline is four steps plus an optional render, and each step saves its output so you can re-run it on its own:
- Parse the PDF into one row per text line, with page number, line number, text, and bounding box, using PyMuPDF. The Attention paper becomes 1048 rows across 15 pages.
- Parse the question into a few literal keywords with a small LLM call. It’s literal on purpose: an earlier prompt asked for 3 to 5 keywords and got paraphrases that appear nowhere in the document.
- Retrieve by counting how many keywords each page contains, using a case-insensitive substring match, and keep the top three. Pages with zero matches are dropped rather than padded.
- Generate the answer as typed JSON: answer, evidence span (start and end page and line), confidence, justification, verbatim quotes, and caveats.
- Optionally draw rectangles over the cited lines using the stored bounding boxes.
Only page numbers cross from retrieval to generation; the lines are rebuilt on the other side. Her stated reason is that an AI assistant told to fix retrieval will quietly edit the question parser instead. It’s the same instinct as saving intermediate output in an import script. Keep the steps independent and you can debug one without breaking the rest.
The schema is the part worth stealing
The generation step forces the model to fill a Pydantic model with the answer, the evidence span, a confidence, a justification, quotes, and caveats. The null path is the part I’d copy first: when the retrieved lines don’t contain the answer, the page and line fields come back null and a caveat says why, instead of the model inventing something plausible.
If you wire a model into a plugin to answer questions about policy pages, shipping terms, or product docs, that contract costs nothing and prevents the worst failure: a confident answer with nothing behind it. OpenAI’s structured outputs enforce the shape at the API level, but in PHP you still verify before rendering anything:
<?php
$answer = bbioon_ask_with_evidence( $question, $context_lines );
// Null path: the model says the lines don't contain the answer.
if ( null === $answer['start_page'] || null === $answer['start_line'] ) {
return new WP_Error(
'bbioon_no_evidence',
$answer['caveats'][0] ?? __( 'No supporting lines were found.' )
);
}
// A schema forces a citation. It doesn't make the citation real.
if ( ! bbioon_quotes_in_lines( $answer['quotes'], $context_lines ) ) {
return new WP_Error( 'bbioon_bad_citation', 'Cited quote missing from the retrieved lines.' );
}
return $answer;The quote check is easy to skip. Compare each quote against the lines you actually sent, the same way you validate any input before acting on it. With WordPress now shipping its own AI plumbing, knowing this contract matters more than knowing any specific plugin built on it.
Keyword retrieval is a defensible first cut
She drops embeddings from the minimal version for a reason I hadn’t seen argued this clearly: a cosine score of 0.78 answers nothing when a client asks why page 9 was picked over page 6. Keyword matching prints the matched terms per page, so anyone can verify the ranking by searching the PDF. It’s the same reason to prefer a WP_Query search you can print over a ranking you can’t explain.
It fails in known ways. Asked for the epsilon used in label smoothing, the pipeline retrieves nothing useful because the paper writes the Greek letter and never the word. On a store, product names and SKUs are literal, so keywords cover most questions. Prose policy pages drift from the question’s vocabulary and need the hybrid mix, which is also where Anthropic lands in its contextual retrieval work.
If you want an answer-with-citations feature on a WordPress or WooCommerce site, small enough to debug, this is the shape I’d start from, and it’s the kind of build I take on.
One warning before you promise anything to a client: the null path says the answer wasn’t in the lines the model saw. It doesn’t prove the answer isn’t in the document. In her World Bank test, the real figure sat on a page the retriever never pulled. Test with questions the docs can’t answer before the ones they can, and keep the feature away from anything that would be read as advice.
