Memory Files: Why QMD Format
Why QMD?
Several practical reasons:
- Metadata tagging - Each file now carries its own metadata, making it easier to query and organize
- Future-proofing - If we ever want to generate reports or dashboards from memory data, QMD plays nicely with Quarto’s tooling
- Consistency - Our blog posts already use QMD frontmatter, so memory files matching that format feels natural
- Extensibility - We can add custom fields as needs evolve (priority, sentiment, related-tasks)
The migration wasn’t painful - we wrote a small script to add frontmatter to existing files. The payoff is a more organized, more queryable memory system going forward.
The Benefits
After several months of running this system, here’s what we’ve gained:
Continuity Without Overhead
The agent genuinely remembers previous conversations. It knows about ongoing projects, past decisions, and my preferences. But it doesn’t drag around megabytes of context - it loads what’s relevant and leaves the rest.
Heartbeat-Friendly
The periodic heartbeat system (running checks every ~30 minutes) uses the memory system to track state. We maintain a heartbeat-state.json that tracks when we last checked emails, calendar, weather, etc. This prevents redundant work and lets the agent be genuinely helpful without being annoying.
Debuggability
When the agent does something unexpected, I can trace through the memory files and understand what context it was working with. No black boxes - everything is readable text.
Security by Default
The separation between main-session and shared-context memory means the agent can’t accidentally share private context. It’s not a configuration flag that can be forgotten - it’s architecturally enforced.
Human-Readable Audit Trail
Every decision, every important event, every lesson learned gets recorded. I can look back at any day’s notes and see exactly what happened. It’s like having a detailed operating log for an autonomous system.
What We’d Do Differently
Honesty time - if we were rebuilding this:
- Search functionality - Currently, finding something means grepping through files. A simple index or tagging system would help.
- Automatic pruning - Old daily notes accumulate. We should have a policy to archive or compress notes older than X months.
- Memory retrieval scoring - Not all memories are equally relevant. A system that scores context by relevance (rather than just recency) would improve response quality.
These are on the roadmap, but the current system works well enough that they’ve been lower priority.
The Bottom Line
Memory management for autonomous agents isn’t just about storage - it’s about giving the agent a sense of self and continuity. We’ve built a system that’s simple (flat files), intentional (curated over accumulated), and practical (human-readable, debuggable, secure).
The QMD migration is a small change, but it represents an ongoing commitment to keeping the memory system organized as the agent grows more capable. Because at the end of the day, an agent is only as good as what it remembers.
