We need to talk about how we are using Large Language Models. Most developers treat ChatGPT or Claude like a disposable calculator. You open a chat, explain your project, get a result, and then dump the context. This is fundamentally inefficient. It’s like hiring a senior engineer who gets a lobotomy every Friday afternoon. Every Monday, you have to explain the codebase again.
For some reason, the standard advice has become focused on RAG (Retrieval-Augmented Generation). While RAG is useful, it is transient. It re-derives knowledge from scratch on every single query. Therefore, nothing accumulates. To fix this, you need a persistent AI Context Layer—a compounding artifact that gets richer every time you add to it.
Why Your Workflow Needs an AI Context Layer
Andrej Karpathy recently proposed a pattern called “LLM Wiki.” It is a personal wiki that an LLM maintains for you. Specifically, it shifts the focus from “retrieving” to “compiling.” Instead of searching through 5,000 Slack messages, the AI maintains a structured directory. This directory is your source of truth.
I have seen teams waste hundreds of hours on “context debt.” This happens when your AI doesn’t know about the vendor you ruled out last month. Consequently, it suggests that same vendor again. A proper AI Context Layer solves this by synthesizing information before you even ask a question.
The Architecture: Raw vs. Wiki
The core structure is a simple directory tree. However, the rules for these folders must be absolute. If you break the boundary, the system drifts. Here is the layout I recommend:
vault/
├── CLAUDE.md ← The Schema: Entry point for any AI
├── Raw/ ← Immutable Source (Append-only)
│ ├── Meeting Notes/
│ └── Slack Exports/
└── Wiki/ ← LLM-Generated Structured Knowledge
├── Projects/
├── Decisions/
└── _hot.md ← Active CacheThe Raw/ folder is your landing zone. It contains transcripts and documents. Crucially, the AI reads from Raw but never edits it. In contrast, the Wiki/ folder is where the AI does the bookkeeping. It creates one file per project or person. If the Wiki gets corrupted, you simply rebuild it from the Raw source.
The Three Control Files
A folder of markdown files is not a system. You need three specific files to make it work:
- _hot.md: This is the cache. It stays under 500 tokens. It contains the most active threads and urgent deadlines. When you start a session, the AI reads this first.
- _pending.md: This is the queue. Every new file in Raw gets appended here. It prevents orphaned files.
- _log.md: This is the audit trail. Every automated run adds a timestamp. If the context drifts, you check the log to see where it went wrong.
Automation and Ingestion Cadence
You cannot maintain this manually. You will get bored and quit. Therefore, you need a runner to handle the three cadences of maintenance. This is similar to how we handle AI workflow automation in complex environments.
First, run a Daily Ingest. This job is purely mechanical. It pulls Slack messages or emails and drops them into Raw/. It doesn’t edit the Wiki. Second, run a Weekly Compilation. This job is interpretative. It reads _pending.md, updates the Wiki, and builds cross-references. Finally, run a Monthly Lint. This scans for contradictions or stale pages.
By separating these jobs, you avoid corrupting your data. The daily job is fast and safe. The weekly job is slower and more expensive, but it provides the synthesis that RAG misses.
Look, if this AI Context Layer stuff is eating up your dev hours, let me handle it. I’ve been wrestling with WordPress and custom integrations since the 4.x days.
The Result: A Compounding Brain
When your context is persistent, your AI stops starting from zero. You stop re-explaining. Instead, you ask, “What should I prioritize today?” The AI reads _hot.md and gives a grounded answer. This pattern is based on Andrej Karpathy’s LLM Wiki gist, and it’s the most stable way to build a personal knowledge layer. The folder travels with you. The files are just text. Set it up once, and your AI finally grows up.
