The default advice for building an AI assistant is now “just throw it in Pinecone.” For a few hundred personal notes or a small side project, that is a logistics fleet delivering one pizza. The Google Memory Agent Pattern is a reaction to that over-engineering, and it is the kind of architecture I actually want to maintain.
I thought I had seen every way a RAG (Retrieval-Augmented Generation) pipeline can fail: cosine similarity returning results that are technically similar and useless in context, or the overhead of babysitting a local Chroma instance for a few hundred Markdown files. Then the math changed. With a 200K context window like Claude Haiku 4.5, you do not need a vector database to find the needle. Structure the haystack well enough and you can hand the model all of it.
The problem with vector-first thinking
Vector search solved a real problem in the era of 4K and 8K token limits. Your data did not fit in the prompt, so you fetched snippets instead. The price was an embedding pipeline, a similarity search API, and amnesia whenever the retriever grabbed the wrong chunk.
Ask “What did Alice say about the budget last February?” and a vector search goes hunting for “Alice” and “budget.” It will probably miss the meeting where Alice only nodded at a spreadsheet. The Google Memory Agent Pattern hands the LLM structured memories instead of raw chunks, so retrieval becomes something the model reasons about rather than a distance calculation.
Context has its own failure mode, which I got into in my post on the Bits-over-Random Metric.
How the Google Memory Agent Pattern works
You store structured metadata in a SQLite database instead of raw text. Three sub-agents move a note through its life:
- IngestAgent: takes raw input such as an Obsidian note, pulls out entities, topics and an importance score, then writes the row to SQLite.
- ConsolidateAgent: the sleeping brain. It scans unconsolidated rows on a schedule, asks the LLM what connects them, and writes what it finds to a separate insights table.
- QueryAgent: on a question, it loads the 50 most recent memories and the top 10 consolidations straight into the context window.
A stripped-down version of the ingest logic in Python, which you can wire to AWS Bedrock or to the official Google repo:
# The bbioon_ingest logic for structured memories
import sqlite3
import json
def bbioon_save_memory(summary, entities, topics, importance):
conn = sqlite3.connect('memory.db')
cursor = conn.cursor()
# We use SQLite because it's portable and doesn't need a Docker container
cursor.execute('''
INSERT INTO memories (summary, entities, topics, importance, consolidated)
VALUES (?, ?, ?, ?, 0)
''', (summary, json.dumps(entities), json.dumps(topics), importance))
conn.commit()
conn.close()The consolidation loop and the amnesia problem
The consolidation pass is where the Google Memory Agent Pattern stops being a plain database. Most memory systems are storage with a nicer label. Here the agent goes back over what it already holds: given three separate notes about a budget worry, the ConsolidateAgent writes a row of its own, “Recurring theme detected: Alice is worried about Q3 overhead.”
I have this hooked up to an Obsidian file watcher. Every 30 minutes a background script computes SHA256 hashes of the files. Mine is a shell script, though in a WordPress context it would be a WP-CLI command. Anything that changed gets its memory rebuilt, so duplicate and stale rows never pile up, and the whole store is one SQLite file I can back up with a copy.
If plumbing an agent memory layer together is eating the dev hours you meant to spend building, hand it over to me. I have been doing WordPress and custom AI integrations since the early days.
When SQLite is enough
Enterprise-scale tools are a poor fit for personal-scale problems. If your data fits in a few thousand SQLite rows and your model has a large context window, the Google Memory Agent Pattern will be faster, cheaper and more accurate than the vector DB setup you would build this weekend. Ship the simple version and see what actually breaks.
