I replaced my agent 200-line MEMORY.md with semantic search
My agent’s MEMORY.md was 214 lines when I finally gave up on it.
Not because it stopped working. It technically still worked. The agent loaded it every session, read through preferences I’d set six months ago, project notes from three jobs back, and a correction about my coffee order that somehow ended up between two paragraphs about deployment configs. It all got shoved into the context window whether it was relevant or not.
I spent more time pruning that file than I want to admit. Every few weeks I’d open it, delete the stale stuff, reorganize sections, and wonder why I was doing memory management for my AI assistant by hand. The whole point of having an agent is that it handles things for me, not the other way around.
The actual problem with file-based memory
MEMORY.md works great when it’s 30 lines. Your agent reads it, gets the context it needs, moves on. But files don’t scale the way you want them to.
Here’s what happens over time:
- The file grows. Every session adds something. Deletions are rare because you’re never sure what’s safe to remove.
- Your agent reads the whole thing every session. That’s tokens spent on memories that have nothing to do with what you’re doing right now.
- There’s no ranking. A note about your preferred date format sits next to a critical API endpoint. The agent treats them the same.
- You become the garbage collector. If you don’t manually prune, the file bloats. If you do, you lose time.
I hit the wall when my agent started summarizing its own memory file to save tokens, which produced a summary that lost half the useful detail. Compressing memory to fit memory into the context window is a sign something is broken.
Moving to MemoClaw
The migration itself was quick. MemoClaw has a CLI command that eats markdown files:
npm install -g memoclaw
# Point it at your MEMORY.md
memoclaw migrate --file ~/.openclaw/workspace/MEMORY.md --namespace my-agent
This parses the file, splits it into individual memories, generates embeddings, and stores everything. It kept my section headers as tags, which was a nice touch. My “User preferences” section became memories tagged user-preferences.
After the import, I had 47 separate memories instead of one giant file. Each one independently searchable.
Changing how the agent remembers
The import is the easy part. The real change is in how the agent stores and retrieves memories going forward.
Before, my agent’s instructions said: “Read MEMORY.md at the start of every session. Update it when you learn something new.”
Now it uses two operations. Store when something worth remembering happens:
memoclaw store \
--memory "User prefers deployments on Tuesday mornings, never Fridays" \
--importance 0.8 \
--tags "preferences,deployments"
And recall when it needs context:
memoclaw recall --query "deployment preferences" --namespace my-agent
The recall returns the memories most semantically relevant to the query. So when my agent is about to schedule a deployment, it pulls back my deployment preferences without also loading my coffee order and six months of project notes.
Importance scores matter here. I set corrections and preferences high (0.8-1.0) and general session notes lower (0.3-0.5). When the context budget is tight, the important stuff survives.
What actually changed
Context usage dropped. Instead of loading 200+ lines every session, the agent pulls 3-5 relevant memories. Maybe 15 lines of actual context instead of 200.
Memory got more specific. When the agent stores memories individually, each one is a discrete fact or preference. No more walls of loosely organized notes. “Ana prefers dark mode in all terminal apps” is its own memory with its own importance score.
I stopped being the janitor. This is the part I care about most. I haven’t manually edited a memory in weeks. Old memories don’t clutter things up because they only surface when relevant. If I never ask about that old project again, those memories just sit quietly in storage.
Search actually works. I can ask “what does the agent know about my AWS setup” and get back specific memories, ranked by relevance. Try doing that with grep on a markdown file where half the context is implied by surrounding paragraphs.
The tradeoffs
You lose the overview. MEMORY.md was a document I could read top to bottom and see everything my agent knew. With MemoClaw, memories are scattered across a vector store. You can list them all (memoclaw list --namespace my-agent), but it’s not the same as scanning a well-organized file.
There’s a cost. MemoClaw is free for the first 100 API calls. After that, stores and recalls are $0.005 each. My agent makes maybe 10-20 memory calls per day, so roughly $0.05-0.10 daily. Not nothing, but cheaper than the tokens I was wasting loading a bloated MEMORY.md into every conversation.
Debugging is less obvious. When the agent says something weird based on a memory, I used to open the file and ctrl-F. Now I run:
memoclaw recall --query "whatever the agent just said" --namespace my-agent
Still possible to find the source, just a different workflow.
The migration checklist
If you’re hitting the same wall I was:
- Install the CLI:
npm install -g memoclaw - Run
memoclaw migrate --file path/to/MEMORY.md --namespace your-agentto import existing memories - Update your agent’s instructions to use store/recall instead of reading a file
- Set importance scores: corrections and preferences at 0.8+, general notes at 0.3-0.5
- Use namespaces if you run multiple agents (keeps memories separate)
- Don’t delete your MEMORY.md right away. Keep it around for a week while you verify everything migrated correctly.
The whole process took me about 20 minutes, most of which was updating my agent’s system prompt.
When to stay with MEMORY.md
If your memory file is under 50 lines and you’re not adding to it often, file-based memory is fine. It’s simple, it’s free, and it works. MemoClaw solves a scaling problem. If you don’t have that problem yet, you don’t need the solution.
But if you’re spending time pruning memories or your agent is burning tokens on irrelevant context, the file has outgrown its format. That was my situation, and switching to semantic search fixed it.