# The Memvid Bridge: A 49% Token Reduction Mystery

*How I solved the case of the vanishing tokens and built an MCP bridge that saved half my context window*

---

## 📖 Table of Contents

- [Chapter 1: The Crime Scene](#chapter-1-the-crime-scene)
- [Chapter 2: The Investigation](#chapter-2-the-investigation)
- [Chapter 3: The First Breakthrough](#chapter-3-the-first-breakthrough)
- [Chapter 4: The Bridge Construction](#chapter-4-the-bridge-construction)
- [Chapter 5: The Memory Restructure](#chapter-5-the-memory-restructure)
- [Chapter 6: The Skills Compression](#chapter-6-the-skills-compression)
- [Chapter 7: The Final Pieces](#chapter-7-the-final-pieces)
- [Chapter 8: The Evidence](#chapter-8-the-evidence)
- [Chapter 9: The Future](#chapter-9-the-future)

---

## Chapter 1: The Crime Scene

It was a Tuesday morning when I first noticed something was wrong. My AI assistant, Hermes, was acting sluggish. Responses were slower, context was getting lost, and the token meter was spinning like a roulette wheel in Vegas.

I pulled up the diagnostics. The numbers didn't lie:

| Metric | Value |
| :-- | :-- |
| [SOUL.md](http://SOUL.md) | 7,044 characters |
| Skills L0 Index | 12,470 characters |
| [MEMORY.md](http://MEMORY.md) | 1,297 characters |
| **TOTAL BASELINE** | **21,424 characters (~5,356 tokens)** |

Over five thousand tokens. Gone. Every single session. Before I even asked a question.

> *"The perfect crime," I muttered to myself. "Someone's stealing half my context window, and they're doing it right under my nose."*

The suspects were obvious:

> 🔍 **CLUE: Suspect #1:** [**SOUL.md**](http://SOUL.md)\
> 7,044 characters of identity, mission, permissions, workflow rules, model selection, code quality standards, focus domains, memory rules, and failure modes. All loaded. Every session. Whether I needed them or not.

> 🔍 **CLUE: Suspect #2: Skills L0 Index**\
> 12,470 characters listing 100 skills with full descriptions. "compressed-skills" with its 234-character description. "memvid-on-demand" with its 312-character description. All there. All the time.

> 🔍 **CLUE: Suspect #3:** [**MEMORY.md**](http://MEMORY.md)\
> 1,297 characters of user preferences, project notes, and random facts. A grab bag of information, some relevant, most not.

I needed a plan. I needed to catch the thief in the act. But first, I needed to understand *how* they were getting away with it.

---

## Chapter 2: The Investigation

I started digging. The Hermes Agent codebase was a labyrinth of Python files, but I found the smoking gun in `agent/prompt_builder.py`:

```python
def build_skills_system_prompt(skills: List[Skill]) -> str:
    """Build the skills system prompt with full descriptions."""
    lines = ["## Available Skills"]
    for skill in skills:
        lines.append(f"- {skill.name}: {skill.description}")
    return "\n".join(lines)
```

There it was. The `skill.description` field. Every single skill, fully described, loaded into the system prompt. No filtering. No on-demand retrieval. Just brute force.

> ⚠️ **OBSTACLE #1: The Hardcoded Prompt**\
> The skills rendering logic was baked into the core codebase. I couldn't change it without modifying Hermes itself. And Hermes updates would overwrite my changes.

I looked at [SOUL.md](http://SOUL.md). Same story. The entire file was loaded at session start. No conditional loading. No retrieval hints. Just everything, all the time.

> ⚠️ **OBSTACLE #2: The Monolithic Memory**\
> All my memories were in a single `memory.mv2` file. Session summaries, project notes, tool configs, provider settings—all mixed together. No way to retrieve just what I needed.

But then I found something interesting. A tool called `memvid`—a CLI for fast local memory retrieval. It could search through `.mv2` files in milliseconds. It had lexical search, semantic search, even natural language question answering.

> 🔍 **THE BREAKTHROUGH:**\
> Memvid was already installed at `~/.hermes/node/bin/memvid`. It was fast. It was local. It was exactly what I needed. But Hermes couldn't use it. There was no integration.

The pieces were coming together. I needed a bridge. A way for Hermes to talk to memvid. A way to load only what was needed, when it was needed.

I opened a terminal and typed:

```bash
mkdir ~/hermes-memvid-bridge
cd ~/hermes-memvid-bridge
uv init
```

The investigation had begun.

---

## Chapter 3: The First Breakthrough

I started with the basics. I needed to understand how memvid worked. I ran a few test commands:

```bash
# Create a test memory file
~/.hermes/node/bin/memvid create test.mv2

# Add a frame
~/.hermes/node/bin/memvid put test.mv2 \
  --title "Test Frame" \
  --tag "type=test" \
  --input test.txt

# Search for it
~/.hermes/node/bin/memvid find test.mv2 --query "test" --json
```

The output was beautiful:

```json
{
  "hits": [
    {
      "frame_id": 0,
      "title": "Test Frame",
      "text": "This is test content...",
      "score": 1.0
    }
  ]
}
```

Fast. Reliable. JSON output. This was going to work.

Now I needed to wrap this in Python. I created `memvid_client.py`:

```python
import subprocess
import json
from typing import Optional, List, Dict, Any

TIMEOUT = 30
TIMEOUT_LONG = 60

def memvid_find(
    store_path: str,
    query: str,
    top_k: int = 5,
    mode: str = "auto"
) -> List[Dict[str, Any]]:
    """Search a memvid store and return matching frames."""
    cmd = [
        "~/.hermes/node/bin/memvid",
        "find",
        store_path,
        "--query", query,
        "--top-k", str(top_k),
        "--mode", mode,
        "--json"
    ]
    result = subprocess.run(
        cmd, capture_output=True, text=True, timeout=TIMEOUT
    )
    if result.returncode != 0:
        raise RuntimeError(f"memvid find failed: {result.stderr}")
    data = json.loads(result.stdout)
    return data.get("hits", [])
```

I tested it. It worked. But I had a problem: how would Hermes know which store to search? I needed a mapping system.

I created `store_resolver.py`:

```python
import os
from pathlib import Path

DEFAULT_ROOT = os.path.expanduser("~/.hermes/memvid")

def resolve_store_path(store: str, root: str | None = None) -> Path:
    """Resolve a named store to its .mv2 file path."""
    if not store or not store.strip():
        raise ValueError("Store name cannot be empty")
    
    # Sanitize: no path traversal
    normalized = store.replace("\\", "/").strip("/")
    if ".." in normalized.split("/"):
        raise ValueError(f"Store name cannot contain '..': {store}")
    
    root = root or os.environ.get("MEMVID_ROOT", DEFAULT_ROOT)
    path = Path(root) / f"{normalized}.mv2"
    return path
```

Now I could do things like:

```python
>>> resolve_store_path("memories")
PosixPath('/home/haryn/.hermes/memvid/memories.mv2')

>>> resolve_store_path("projects/language-line")
PosixPath('/home/haryn/.hermes/memvid/projects/language-line.mv2')
```

The foundation was laid. Now I needed the bridge itself.

---

## Chapter 4: The Bridge Construction

The Model Context Protocol (MCP) was the key. It allowed Hermes to expose tools to external servers via stdio. I created `bridge.py`:

```python
from mcp.server.stdio import stdio_server
from mcp.server import Server
from mcp.types import Tool, TextContent
import json
from store_resolver import resolve_store_path, get_memvid_bin
from memvid_client import (
    memvid_find, memvid_put, memvid_update,
    memvid_ask, memvid_stats, memvid_timeline
)

app = Server("memvid-bridge")

@app.tool()
def memvid_find_tool(
    store: str, query: str, top_k: int = 5, mode: str = "auto"
) -> list[TextContent]:
    """Search a named memvid store."""
    store_path = resolve_store_path(store)
    hits = memvid_find(str(store_path), query, top_k, mode)
    return [TextContent(
        type="text",
        text=json.dumps(hits, indent=2)
    )]

@app.tool()
def memvid_put_tool(
    store: str, title: str, text: str,
    label: str | None = None,
    tags: list[str] | None = None,
    metadata: dict | None = None
) -> list[TextContent]:
    """Store a new frame in a named memvid store."""
    store_path = resolve_store_path(store)
    result = memvid_put(str(store_path), title, text, label, tags, metadata)
    return [TextContent(
        type="text",
        text=f"Stored frame with ID: {result['frame_id']}"
    )]

# ... more tools for update, ask, stats, timeline ...

async def main():
    async with stdio_server() as (read_stream, write_stream):
        await app.run(
            read_stream,
            write_stream,
            app.create_initialization_options()
        )

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())
```

I tested it. I ran it. It worked. But I had obstacles to overcome.

> ⚠️ **OBSTACLE #3: The Missing Store**\
> What if someone tried to search a store that didn't exist? The memvid CLI would error out. I needed to handle this gracefully.

> ✅ **SOLUTION:**\
> I added error handling in the client functions. If a store doesn't exist on read, return empty results. If it doesn't exist on write, create it first.

> ⚠️ **OBSTACLE #4: The Timeout**\
> Some memvid operations took longer than others. The default 30-second timeout wasn't enough for large stores.

> ✅ **SOLUTION:**\
> I extracted the magic number into a constant `TIMEOUT_LONG = 60` for operations that need more time.

I wrote tests. Lots of tests:

```python
# test_store_resolver.py
def test_resolve_flat_store():
    path = resolve_store_path("memories")
    assert path == Path.home() / ".hermes/memvid/memories.mv2"

def test_resolve_nested_store():
    path = resolve_store_path("projects/language-line")
    assert path == Path.home() / ".hermes/memvid/projects/language-line.mv2"

def test_path_traversal_protection():
    with pytest.raises(ValueError):
        resolve_store_path("../etc/passwd")

# test_memvid_client_find.py
def test_find_with_mocked_subprocess(mocker):
    mock_result = mocker.Mock()
    mock_result.returncode = 0
    mock_result.stdout = '{"hits": [{"frame_id": 0, "title": "Test"}]}'
    mocker.patch("subprocess.run", return_value=mock_result)
    
    hits = memvid_find("/tmp/test.mv2", "test")
    assert len(hits) == 1
    assert hits[0]["title"] == "Test"
```

60 tests. All passing. The bridge was ready.

---

## Chapter 5: The Memory Restructure

Now came the hard part. I had to take my monolithic `memory.mv2` and split it into purpose-built stores. I created a migration plan:

| Store File | Content | Source |
| :-- | :-- | :-- |
| `identity.mv2` | [SOUL.md](http://SOUL.md) procedural rules | [SOUL.md](http://SOUL.md) sections 6-12 |
| `user-profile.mv2` | User preferences, communication style | [USER.md](http://USER.md) + [MEMORY.md](http://MEMORY.md) user entries |
| `memories.mv2` | Session summaries, lessons, decisions | [MEMORY.md](http://MEMORY.md) + memory.mv2 frames |
| `skills-index.mv2` | Skill descriptions, usage patterns | All 100 [SKILL.md](http://SKILL.md) files |
| `providers.mv2` | LLM provider configs, endpoints | memory.mv2 provider entries |
| `tools.mv2` | Tool definitions, API endpoints | memory.mv2 tool entries |
| `scripts.mv2` | Saved scripts, automation recipes | memory.mv2 script entries |
| `projects/language-line.mv2` | LanguageLine project context | memory.mv2 LanguageLine frames |

I wrote a script to extract all skill descriptions:

```python
import os
import json
from pathlib import Path

skills_dir = Path.home() / ".hermes/skills"
skills_data = []

for category_dir in skills_dir.iterdir():
    if not category_dir.is_dir():
        continue
    category = category_dir.name
    for skill_file in category_dir.glob("*/SKILL.md"):
        skill_name = skill_file.parent.name
        with open(skill_file) as f:
            content = f.read()
            
        # Extract frontmatter
        if content.startswith("---"):
            _, frontmatter, _ = content.split("---", 2)
            metadata = yaml.safe_load(frontmatter)
            skills_data.append({
                "name": skill_name,
                "category": category,
                "description": metadata.get("description", ""),
                "tags": metadata.get("tags", [])
            })

with open("/tmp/skills_data.json", "w") as f:
    json.dump(skills_data, f, indent=2)
```

109 skills. 23 categories. All extracted. Now I ingested them into `skills-index.mv2`:

```python
for skill in skills_data:
    memvid_put(
        store_path="~/.hermes/memvid/skills-index.mv2",
        title=f"{skill['category']}/{skill['name']}",
        text=skill['description'],
        tags=[f"category={skill['category']}", f"skill={skill['name']}"]
    )
```

I did the same for the other stores. I backed up the original `memory.mv2` to `memory.mv2.bak`. I verified retrieval worked.

> ⚠️ **OBSTACLE #5: The Tag Format**\
> I kept getting errors when trying to add tags. The memvid CLI expected `--tag KEY=VALUE`, but I was passing `--tag VALUE`.

> ✅ **SOLUTION:**\
> I read the help output carefully and fixed the format. Tags now look like `--tag "category=software-development"` and `--tag "skill=compressed-skills"`.

The restructure was complete. I had 8 purpose-built stores. Each one focused. Each one fast to search.

---

## Chapter 6: The Skills Compression

Now for the big one. The Skills L0 Index. 12,470 characters of skill descriptions. I needed to compress it.

My first thought was to just strip the descriptions:

```python
def build_skills_system_prompt(skills: List[Skill]) -> str:
    lines = ["## Available Skills"]
    for skill in skills:
        lines.append(f"- {skill.name}")  # No description!
    return "\n".join(lines)
```

But that wouldn't work. How would I know what a skill did? I needed a way to retrieve descriptions on demand.

I created a new skill: `compressed-skills`. It would teach Hermes how to retrieve skill descriptions:

````yaml
---
name: compressed-skills
description: Retrieve skill descriptions from skills-index.mv2 on demand
tags: [skills, retrieval, memvid]
---

# Compressed Skills Retrieval

The skills L0 index contains only skill names and categories. Full descriptions are stored in `skills-index.mv2`.

## When to Use

When you need to understand what a skill does before using it:
1. Use `memvid_find(store="skills-index", query="<skill-name>")` to get the description
2. Use `skill_view(name="<skill-name>")` to get the full skill content

## Example

```text
# User asks: "What does the compressed-skills skill do?"

# Step 1: Retrieve description
memvid_find(store="skills-index", query="compressed-skills")
# Returns: "Retrieve skill descriptions from skills-index.mv2 on demand"

# Step 2: Load full skill
skill_view(name="compressed-skills")
# Returns: Full SKILL.md content
```
````

But I still had a problem: the `build_skills_system_prompt()` function was hardcoded in Hermes core. I couldn't change it without modifying the codebase.

> ⚠️ **OBSTACLE #6: The Core Code**\
> The skills rendering logic was in `~/.hermes/hermes-agent/agent/prompt_builder.py`. Modifying it would break on the next `hermes update`.

> ✅ **SOLUTION:**\
> I created a patch file. I would apply it after each `hermes update`. The patch would strip descriptions from the skills prompt and add a retrieval hint.

I created `patches/001-skills-l0-compression.patch`:

```diff
--- a/agent/prompt_builder.py
+++ b/agent/prompt_builder.py
@@ -45,7 +45,7 @@ def build_skills_system_prompt(skills: List[Skill]) -> str:
     lines = ["## Available Skills"]
     for skill in skills:
         category = skill.category or "uncategorized"
-        lines.append(f"- {skill.name}: {skill.description}")
+        lines.append(f"- {skill.name}")
     return "\n".join(lines)
```

I applied the patch:

```bash
cd ~/.hermes/hermes-agent
git apply ~/hermes-memvid-bridge/patches/001-skills-l0-compression.patch
```

I cleared the snapshot cache:

```bash
rm ~/.hermes/.cache/snapshots/*
```

I measured the result:

| Metric | Before | After | Reduction |
| :-- | :-- | :-- | :-- |
| Skills L0 | 12,187 chars | 3,132 chars | **74%** |

It worked. The skills index was now 74% smaller. And I could still retrieve descriptions on demand.

---

## Chapter 7: The Final Pieces

I had the bridge. I had the restructured memory. I had the compressed skills. Now I needed to tie it all together.

First, I updated `MEMORY.md` to be a thin retrieval index:

```markdown
# Memory Index

## Active Context (loaded at session start)
- User: H-Man, WSL2 + Windows, night-owl coder
- Active projects: LanguageLine automation, hermes-memvid-bridge
- Open loops: Codex audit pass, OpenClaw sync, patch re-apply after updates

## On-Demand Retrieval
When you need facts, lessons, decisions, or deep context:
- Identity rules: `memvid__find(store="identity", query="workflow OR model selection OR code quality")`
- Memories: `memvid__find(store="memories", query="session summary OR lesson OR decision")`
- Projects/LanguageLine: `memvid__find(store="projects/language-line", query="LanguageLine OR VoS")`
- Projects/MemvidBridge: `memvid__find(store="projects/memvid-bridge", query="memvid bridge OR token savings OR patch")`
- Providers: `memvid__find(store="providers", query="ollama OR openrouter OR z.ai")`
- Tools: `memvid__find(store="tools", query="environment OR camofox OR MCP")`
- User prefs: `memvid__find(store="user-profile", query="preferences OR style")`
- Skills index: `memvid__find(store="skills-index", query="skill name OR category")`

## Key Facts (high-frequency, always needed)
Ollama: qwen3.5:9b (local, Q4_K_M, 6.5GB). Local: Bonsai 8b/4b (localhost:8080).
Providers: MiniMax M2.7, Z.ai glm-5.1, Gemini flash/pro, DeepSeek v4 (ask first). Free: OpenRouter.
LanguageLine: Cron 303561489a0e (9 AM). Project: ~/language-line-automation/. Skills: language-line-osi.
Bonsai: Systemd bonsai-llama. Models: ~/Bonsai-demo/models/ternary-gguf/.
MCP headers: `Bearer ENV_VAR_NAME` format. Z.ai MCP → fallback to web_search when 500.
Parallel delegation: one model per provider. Codex via terminal(pty=true), never --acp/--stdio.
hermes-memvid-bridge: v0.1.0 delivered. Patch re-apply: cd ~/.hermes/hermes-agent && git apply ~/hermes-memvid-bridge/patches/001-skills-l0-compression.patch

## Last updated: 2026-05-02
```

Then I created the `memvid-on-demand` skill to teach Hermes when to use which retrieval method:

````markdown
---
name: memvid-on-demand
description: Decision tree for memvid vs session_search vs skill_view
tags: [memvid, retrieval, decision-tree]
---

# Memvid On-Demand Retrieval

## Decision Tree

```text
Need to retrieve information?
│
├─ Is it about a specific project?
│  └─ YES → Use memvid__find(store="projects/<project-name>", query="...")
│
├─ Is it a skill description?
│  └─ YES → Use memvid__find(store="skills-index", query="<skill-name>")
│
├─ Is it from a past session?
│  └─ YES → Use memvid__find(store="memories", query="session summary OR lesson")
│
├─ Is it a procedural rule (permissions, workflow)?
│  └─ YES → Use memvid__find(store="identity", query="workflow OR permissions")
│
└─ Is it general knowledge or recent context?
   └─ YES → Use session_search(query="...")
```

## Store Routing Table

| Store | Use For | Example Query |
| :--- | :--- | :--- |
| `identity` | Procedural rules | "workflow", "model selection", "code quality" |
| `memories` | Session summaries | "session summary", "lesson", "decision" |
| `skills-index` | Skill descriptions | "compressed-skills", "memvid-on-demand" |
| `providers` | LLM configs | "ollama", "openrouter", "z.ai" |
| `tools` | Tool definitions | "environment", "camofox", "MCP" |
| `user-profile` | User preferences | "preferences", "style" |
| `projects/*` | Project context | "LanguageLine", "VoS", "memvid bridge" |

## Anti-Patterns
❌ Don't store in MEMORY.md what belongs in a .mv2 file  
❌ Don't use session_search for project-specific context  
❌ Don't use memvid__find for recent session context  
❌ Don't duplicate the same information in multiple stores  
````

Finally, I updated the `session-handoff` skill to write session summaries to `memories.mv2`:

### Step 3: Write Session Summary to Memvid

Store the session summary in `memories.mv2` via the MCP bridge:

```python
memvid__put(
  store="memories",
  title="session:YYYY-MM-DD:topic",
  text="<session summary content — what was done, decisions, files changed, open loops, next steps>",
  tags=["daily-log", "YYYY-MM-DD", "project-name", "topic"]
)
```

### Step 4: Check [MEMORY.md](http://MEMORY.md) Threshold

1. Read `~/.hermes/memories/MEMORY.md`
2. If it exceeds 80% of 2,200 char limit (1,760 chars), regenerate: a. Extract key facts that should stay (active context, store map, high-frequency facts) b. Move detailed entries to `memories.mv2` via `memvid__put` c. Rewrite [MEMORY.md](http://MEMORY.md) using the thin index template

I ran end-to-end tests. I verified retrieval worked. I checked the token budget.

---

## Chapter 8: The Evidence

The results were in. I pulled up the final numbers:

| Metric | Before | After | Reduction |
| :-- | :-- | :-- | :-- |
| Skills L0 | 12,187 chars | 3,132 chars | **74%** |
| [SOUL.md](http://SOUL.md) | 7,044 chars | 5,081 chars | **28%** |
| **TOTAL BASELINE** | **21,141 chars** | **10,813 chars** | **49%** |

49% reduction. Half my context window back. The thief had been caught.

I documented everything in the README:

```markdown
# hermes-memvid-bridge

MCP server bridge between Hermes Agent and memvid CLI.
Exposes multi-file `.mv2` memory stores as on-demand retrieval tools.

Reduces per-session token baseline by ~49% through compressed skills index,
split SOUL.md, and thin MEMORY.md retrieval pattern.

## Token Savings (v0.1.0 Measured)

| Component | Before | After | Reduction |
| :--- | :--- | :--- | :--- |
| Skills L0 | 12,187 chars | 3,132 chars | 74% |
| SOUL.md | 7,044 chars | 5,081 chars | 28% |
| **Total baseline** | **21,141 chars** | **10,813 chars** | **49%** |
```

I tagged the release:

```bash
git tag v0.1.0
git push origin v0.1.0
```

I pushed to GitHub:

```bash
git push origin main
```

The case was closed. But the investigation had taught me things.

---

## Chapter 9: The Future

The mystery was solved, but there were still loose ends. Things I'd learned. Things I'd do differently next time.

### Lessons Learned

> 🔍 **LESSON #1: On-Demand > Always-Loaded**\
> The biggest win was moving from "load everything" to "load what you need." Retrieval is fast (&lt;5ms). Context is expensive. Always prefer on-demand retrieval.

> 🔍 **LESSON #2: Purpose-Built Stores > Monolithic**\
> Splitting memory into focused stores made retrieval faster and more accurate. I could search just project context, or just procedural rules, without noise from other domains.

> 🔍 **LESSON #3: Skills Are Documentation**\
> The skills I created (`compressed-skills`, `memvid-on-demand`) became part of the system's knowledge. They taught future sessions how to use the new architecture.

> 🔍 **LESSON #4: Patches Are Fragile**\
> The patch approach works, but it's fragile. Every `hermes update` requires re-applying the patch. A better solution would be a plugin system or configuration-based override.

### Future Enhancements

> ✅ **ENHANCEMENT #1: Auto-Patch on Update**\
> Detect when `hermes update` runs and automatically re-apply the patch. Could be done with a git hook or a wrapper script.

> ✅ **ENHANCEMENT #2: Codex Deep Audit**\
> Run a comprehensive Codex audit to find deeper issues. The manual audit I did was good, but Codex might find things I missed.

> ✅ **ENHANCEMENT #3: OpenClaw Sync**\
> Implement cross-harness shared memory. Hermes and OpenClaw could share the same `.mv2` stores for consistent context across agents.

> ✅ **ENHANCEMENT #4: Per-Project Stores**\
> Create a new `.mv2` store for each new project. The `projects/`
