The default advice for anyone building an AI-powered tool has become “just install a dedicated RAG vector database like Pinecone or Milvus.” Those tools earn their keep on enterprise systems holding hundreds of millions of vectors. For the documentation bots, internal MVPs and product search boxes most of us actually ship, they are more infrastructure than the problem needs.
Fourteen years into this work, I have watched the pattern repeat. A new tool shows up and every problem starts looking like it needs a distributed cluster. Meanwhile the extra database costs you network latency and serialization time, and hands you one more thing that can fail. Vector search is matrix multiplication, and Python has had excellent tools for that for years.
Why matrix math is enough for now
A RAG workflow has four steps: embed the text into vectors, store them, retrieve the similar ones with cosine similarity, generate a response. That retrieval step is a dot product. Normalize your vectors to a magnitude of 1 and cosine similarity is the dot product, nothing more.
NumPy exists for exactly these operations. Its vectorized routines lean on modern CPU features, so you can search millions of text strings in memory in milliseconds. It is the same argument I made in pragmatic AI workflow automation: pick the simplest tool that solves the problem reliably.
Building a simple vector store with NumPy
A plain Python class can hold your embeddings instead of a server, which keeps the stack lean and the latency low. Here is a production-ready implementation of the basic RAG vector database logic in NumPy:
import numpy as np
from sentence_transformers import SentenceTransformer
class bbioon_SimpleStore:
def __init__(self, model_name='all-MiniLM-L6-v2'):
self.encoder = SentenceTransformer(model_name)
self.documents = []
self.embeddings = None
def add_docs(self, docs):
texts = [d['text'] for d in docs]
new_vecs = self.encoder.encode(texts)
# Normalization: Critical for dot product speed
norm = np.linalg.norm(new_vecs, axis=1, keepdims=True)
new_vecs = new_vecs / norm
if self.embeddings is None:
self.embeddings = new_vecs
else:
self.embeddings = np.vstack([self.embeddings, new_vecs])
self.documents.extend(docs)
def search(self, query, k=5):
query_vec = self.encoder.encode([query])
query_vec = query_vec / np.linalg.norm(query_vec)
# The "Matrix Math" Magic
scores = np.dot(self.embeddings, query_vec.T).flatten()
top_k = np.argsort(scores)[-k:][::-1]
return [{"score": scores[i], "text": self.documents[i]['text']} for i in top_k]This runs offline, with no network hop and nothing external to depend on. Run it as a microservice next to a WordPress site and the overhead barely registers next to calling a third-party API on every search.
Scaling up with SciKit-Learn
When brute-force matrix multiplication does start to drag, somewhere in the hundreds of thousands of documents, the answer still is not a dedicated RAG vector database server. Move up to SciKit-Learn’s NearestNeighbors instead.
Tree structures like KD-Tree and Ball-Tree bring search complexity down to O(log N). In recent tests, a search across 1.2 million lines of text, War and Peace and A Christmas Carol among them, came back in less than a tenth of a second. The official SciKit-Learn documentation explains how the algorithms work.
When you actually need a dedicated RAG vector database
I am not against these tools. There is a tipping point where migrating to Weaviate or pgvector is the right call, and these are the signs of it:
- Rebuilding the index from scratch on every server restart is not an option for you.
- Your embedding matrix outgrows the memory available on the server, though a million 384-dimensional vectors only take about 1.5GB.
- You are updating or deleting individual vectors constantly while the system sits under heavy read load.
- You need relational filters over the results, something like finding vectors near X where user_id=45.
Most of us integrating AI into WordPress or WooCommerce will not hit any of those in the first year of a project. I went through similar performance trade-offs in my post on WordPress AI benchmarks.
If the retrieval side of your project is eating your dev hours, I take on that work. I have been building on WordPress since the 4.x days.
Ship the simple version first
Engineering is trade-offs. Choosing a specialized database before you have the data volume to justify it is technical debt with a monthly invoice attached. Starting with NumPy or SciKit-Learn buys you lower latency, a smaller bill and a codebase you can read, and it spares you the race conditions and network hops that turn debugging into guesswork. Ship the MVP, then scale the infrastructure when the data demands it.
