Context engineering deserves better than the current standard advice, which has settled on “just throw more tokens at the prompt.” What that gets you is bloated requests and stateless agents that greet a returning user like a stranger. It hurts the experience and it is a performance problem. A real LLM Memory Layer has to be built for persistence rather than for the session sitting in front of you.
Plenty of projects fail here because they lean on a simple “chat history” array. An array is a log. Memory means extraction, vectorization, and a maintenance loop that can survive contradictions. If a user says they like tea on Monday and coffee on Tuesday, something has to decide which fact wins.
The architecture of persistent memory
A working LLM Memory Layer does four things on its own: it extracts factoids, embeds them as vectors, retrieves them by relevance, and prunes the database so the data does not go stale. I treat that as a context engineering problem more than a storage problem. My notes on advanced LLM optimization cover where this sits in the bigger picture.
- Extraction: turning messy transcripts into atomic, structured facts.
- Vector DB: storing embeddings with metadata filtering, using something like QDrant.
- Retrieval: pulling only what the current turn needs.
- Maintenance: a ReAct (Reasoning and Acting) loop that updates or deletes facts which no longer hold.
Step 1: extracting factoids with DSPy
DSPy has been far more stable than raw prompt engineering for extraction. You define a signature that forces the model to return a list of atomic strings, which keeps the narrative fluff out and leaves clean data points for the LLM Memory Layer.
import dspy
class MemoryExtract(dspy.Signature):
"""
Extract atomic, independent factoids about the user from the transcript.
If no new information is present, return an empty list.
"""
transcript = dspy.InputField()
memories = dspy.OutputField(desc="List of strings containing independent facts")
# Usage example
extractor = dspy.Predict(MemoryExtract)
transcript_data = "I used to love tea, but now I'm a coffee person. Also, I live in Dubai."
result = extractor(transcript=transcript_data)
# Expected result: ["User likes coffee", "User no longer likes tea", "User lives in Dubai"]Step 2: vector storage with QDrant
Then you have to store the factoids somewhere. I use QDrant because its payload filtering is fast. Along with the vector you save the user_id and a timestamp, which isolates memories per user without spinning up thousands of separate collections.
from qdrant_client import AsyncQdrantClient
from qdrant_client.models import VectorParams, Distance
# Set up the collection with a small, fast embedding dimension
async def setup_memory_db(client):
await client.create_collection(
collection_name="user_memories",
vectors_config=VectorParams(size=64, distance=Distance.DOT),
)Maintenance and the ReAct loop
This is where it usually goes wrong. Developers keep appending new facts, and eventually a vector search hands back three different home addresses for the same person. You need a maintenance agent: it takes the new fact, searches for similar existing ones, and decides whether to ADD, UPDATE, or DELETE.
The design comes from the Mem0 research paper, which treats memory as a dynamic pool instead of a static log. If you are fighting the same logic inside WordPress-based AI work, I keep notes on the WordPress AI experiments I have been running.
If an LLM memory layer is eating your dev hours, hand it to me. I have been wrestling with WordPress and backend integrations since the 4.x days.
Pragmatic takeaway
Statelessness is fine for an API and useless for a user relationship. A dedicated LLM Memory Layer built on DSPy and QDrant cuts token waste and makes personalization real, because the system remembers what a user told you instead of guessing at it. That is worth a refactor of your context layer.
