Giving Claude Code A Persistent Memory

If you use Claude Code regularly, you've probably ran into this same problem. You spend some time in a session teaching it something specific, something crucial to your workflow that you don't want Claude to forget. Then you close that window and and open a new session, and it's gone. Claude completely forgot that really important thing. You're back to explaining it all over again.

Claude Code is stateless across sessions by design. That's just fine for a lot of workflows. But for the kind of sustained, project convention-specific work we do here at Viget (where context is built up over months, not minutes) it starts to feel like working with a brilliant colleague who has no long-term memory.

So I built something to solve that problem for myself. It's called recall, and the core idea is pretty simple. It watches every session, pulls out what's durable knowledge, and feeds it back into Claude the next time you open it in that project. The more you use it, the more it knows. The knowledge keeps compounding instead of evaporating.

It's been running across all my active projects for about six weeks now. It's captured almost a hundred sessions. It's distilled plenty of notes that have helped improved new Claude Code sessions. It's even suggested new skills for me to add based on repeated workflows it noticed. And the whole thing costs around $20/month to run.

Here's how it works, and why I built it the way I did.

The Idea #

Andrej Karpathy has been talking about this concept that caught my attention. He published a gist on the topic he called "LLM Wiki". He talks about the missing third paradigm of LLM Learning. Pre-training gives knowledge, fine tuning gives habits, but there's no mechanism for a model to write explicit lessons for itself and carry them forward. There are quite a few open-source projects chasing this vision. Recall is my own attempt at a practical, slightly opinionated implementation of that paradigm.

The Loop – how it works #

Recall is a fairly simple four stage loop.

1. Capture – a hook fires when any session ends and dumps it to a private vault. Pure shell, no LLM call (free!)
2. Distill – every night, a local job reads the day's undistilled sessions and compresses them into durable knowledge note. It averages around 27x compression. One cheap batch run.
3. Retrieve – when you open Claude in a project, a hook injects the relevant knowledge index as context. Claude then reads individual notes on demand from there. It automatically knows all the new knowledge every time it starts.
4. Surface – when the distiller spots a recurring pattern across sessions, it writes a skill proposal to an inbox. You can then review it and decide if it should be added or not. Nothing happens automatically here (on purpose).

The feedback loop is the key: knowledge written in step 2 is read back in step 3. Every session both feeds to and benefits from the vault. It gets smarter automatically the more you use it.

Design Decisions #

This is where my implementation is a bit more opinionated. I chose to keep the LLM interaction to a minimum to not increase token usage. After all, in my previous post about Using Claude Code More Intentionally, token hygiene is important!

Capture is free by design #

The Claude Code hook does one thing. It dumps the sessions and exits. No LLM call, no summarization, no risk of it being expensive to run (or interfering with your session). The point is that it should grab what you did in that sessions and stay out of the way.

Smart Distillation #

The distill step is done as cheap as possible without sacrificing quality. When it runs, the bash script strips out every tool call and irrelevant info so only the actual conversation is processed. It's also locked to sonnet-4-6 to keep costs down but with the aim to capture the most helpful information.

Index-only retrival #

The retrieve hook doesn't inject full, robust notes. Instead it loads a tiny index, full of links to all of the lessons learned. Claude reads all those individual notes on demand when it needs them. This is the key design feature that keeps per-session token cost to a minimum even as the vault grows to hundreds of notes.

Suggestions over automation #

The distiller can propose new skills, but it doesn't create them automatically. Any recuring pattern will land in inbox/proposals.md with a scope of either project or global as well as a confidence rating. You decide what becomes a skill or doesn't. It's not only a safety measure, but it helps keep token costs down as well but not allowing skills to be created without your knowledge that can be token hungry.

Knowledge is scoped as narrowly as possible #

A pattern in one project stays local to that project. A pattern that shows up across two or more projects gets proposed as global knowledge. This keeps your global knowledge lean and your project knowledge relevant.

How it's doing #

I've been running this for about six weeks now and can honestly say it's been a game changer for me. I love when I'm working in a session and know that something I worked on in another session is remembered. Sure enough, I'll see Claude retrieve data from my vault and automatically remember the thing I needed. No explaining it again! Did I mention there's also a dashboard to monitor how things are going? It updates after every distill so you're always in the loop with what's happening. In the six weeks, it's managed to:

• Capture over 80 sessions across 6 projects
• Distill over 70 knowledge notes
• Compress the session data by ~27x
• Use only ~$0.85/nightly run (projected at ~$20/month – and this could be even lower if I were to use Haiku)
• Propose 7 skills

I've even shipped one of the skills. Recall noticed that a few times when I had multiple PRs open and they diverged from the default branch that I was going through and rebasing all of them. So it helped me ship a skill to rebase all my open PRs in the current project based on that repo's default branch. It was a great suggestion!

Where does this fit into the agentic OS landscape? #

There's a lot of movement right now around so-called agentic operating systems. Entire systems built to orchestrate, monitor and automate. This is not trying to be that. It was a the solution to the small problem I face with the least friction. It's also portable + sharable. If I move to a new machine or want to share with my team, my vault is synced to git and easy to retain. It uses existing tooling – Claude Code hooks (Stop + SessionStart), some bash + jq, a call to claude -p headless to distill the compressed session data, launchd for scheduling (macOS only for the moment), and a folder of markdown as the vault. No servers, no database (although there's Obsidian support for the vault!), nothing to keep running. Set it and forget it let Claude remember things.

The compounding thesis #

So the problem is simple. A stateless assistant that resets every session can sometimes be helpful. But for long-running projects (and sessions) you need it to remember conventions, decisions, gotchas, etc. Recall turns one-off discoveries into a permanent, growing knowledge base. The longer it runs, the wider the gap versus starting from zero. Claude really starts to feel more like a brilliant colleague with deep knowledge of your project and team's crucial information.

If you're curious to learn more about recall, it's available on GitHub. Let me know if you've faced a similar problem, tried other solutions or what's working for you!