The default advice for feeding data to an LLM is to dump a pile of PDFs into a vector store and hope. That is where the performance goes. An Efficient Knowledge Base is a curated set of documents, not a landfill.
After 14 years of database bottlenecks and legacy code, the pattern is familiar. It is the same magic-bullet thinking that ends in 40-second queries on a bloated WooCommerce store. The model hallucinates because the retrieval layer handed it garbage. Below is how I put a retrieval-augmented generation (RAG) system together instead.
1. Curate over collect: the value vs. volume trap
The first mistake is assuming more data means a better model. An Efficient Knowledge Base runs on a smaller set of accurate documents. A support bot needs the current policy docs, not every Slack thread from 2019. Irrelevant documents add noise to the vector space and burn tokens on context nobody asked about. I have watched teams spend thousands in API fees because they never pruned the dataset.
What you send matters as much as what you store. I went through context payload optimization separately, and it is most of how you keep token usage down without losing accuracy.
2. Chunking and metadata strategy
Clean data still has to be segmented, and splitting on character count is the lazy way to do it. Chunk by meaning: if a document explains how to reset a password, that whole explanation is one chunk. Metadata is your wp_postmeta here. It lets the system filter candidates before it spends anything on vector similarity.
import numpy as np
def bbioon_prepare_record(doc_id, embedding, text, source, extra_metadata=None):
# Normalize vectors to ensure dot-product behaves like cosine similarity
arr = np.array(embedding, dtype=np.float32)
norm = np.linalg.norm(arr)
normalized_values = (arr / norm).tolist() if norm > 0 else embedding
metadata = {
"source": source,
"char_count": len(text),
"text_preview": text[:500]
}
if extra_metadata:
metadata.update(extra_metadata)
return {
"id": doc_id,
"values": normalized_values,
"metadata": metadata
}3. Vector quantization for faster retrieval
Retrieval latency is the part users feel, and nobody enjoys watching a “typing…” animation. Product Quantization (PQ) or Scalar Quantization compresses those 32-bit floats into smaller representations. The official Pinecone documentation covers how much memory that saves. Smaller vectors read off disk faster and compare faster.
4. Hybrid retrieval: why vector search is not enough
Embeddings are good with concepts and bad at exact strings like a SKU or an error code. Hybrid retrieval covers the gap: run BM25 keyword search alongside vector similarity and merge the two with Reciprocal Rank Fusion (RRF). When someone types “Error E0427” the system returns that exact ID rather than something loosely related to billing errors.
def bbioon_reciprocal_rank_fusion(*ranked_lists, k=60):
# Standard RRF implementation for merging keyword and vector results
from collections import defaultdict
rrf_scores = defaultdict(float)
for ranked_list in ranked_lists:
for rank, (doc_id, _) in enumerate(ranked_list, start=1):
rrf_scores[doc_id] += 1.0 / (k + rank)
return sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)Measured against dense-only retrieval, hybrid search models bring back noticeably more of the relevant documents.
5. Automatic quality monitoring
A knowledge base is not a set-and-forget project. Run automated tests on retrieval quality with a framework like DeepEval. When the bot starts answering from stale documents, you want to hear about it before your customers do. Score interactions on Faithfulness, which catches hallucinations, and on Answer Relevancy.
That evaluation layer costs compute, and the savings elsewhere pay for it. I broke down where agentic AI token savings come from in an earlier post.
If this kind of retrieval work is eating your dev hours, I can take it on. I have been working with WordPress since the 4.x days, and I build systems that scale without eating the whole cloud budget.
Summary: the curated asset mindset
Treat the data as something you design. An efficient knowledge base comes down to clean source documents, chunks that follow the meaning, hybrid retrieval, and monitoring that keeps running after launch. Do that and the answers stop being guesses.
