I built a memory layer that works across Claude, ChatGPT and Cursor — and rendered it as a 3D brain

Every AI tool remembers you a little. None of them remember you to each other. The context I build up in ChatGPT is invisible to Claude, and my Cursor sessions know nothing about either. And the worst version of it: having a genuinely useful long conversation, then watching it sink into an infinite sidebar where I'd never find it again. So I spent the last stretch building Mind Silo — one memory layer that all of them read and write to.

How it works

Capture. A Chrome MV3 extension passively reads conversations on Claude, ChatGPT, Gemini, Cursor and Claude Code. No copy-paste step, no "save this" button to remember to press.

Extraction. Raw transcripts are useless as memory — they're too long, too redundant, and semantically noisy. A background worker (Redis/Arq) distills each conversation into atomic memories: single facts, decisions, and preferences that stand on their own.

Storage. Postgres with pgvector. Each memory gets an embedding, and semantically related memories get connected into a graph rather than sitting in a flat list.

Retrieval. An MCP server (mind-silo-mcp on npm) exposes three tools, so any MCP-compatible client — Claude Desktop, Claude Code, Cursor — can query the memory graph directly and write new memories back to it.

Encryption. Per-user AES-256 with derived keys, which means deletion is a crypto-shred: destroy the key and the data is genuinely unrecoverable, not just flagged deleted in a table.

The part I didn't expect to matter

I built the 3D brain visualization (Three.js) as a fun weekend thing. Memories render as nodes in 3D space, semantic connections draw as edges, projects cluster, topics get their own colors.

It turned out to be the thing people actually react to. Every competitor in this space renders memory as a text list or a set of buckets, and a text list of 400 memories is not something anyone looks at twice. A graph you can fly around is. The visualization ended up being how people understand what the product even does.

Stack

FastAPI (Python 3.12, uv) · Next.js 14 · Three.js · Postgres + pgvector · Redis/Arq · Clerk · Stripe · pnpm monorepo · Render + Vercel

What I'm still figuring out

The hard problem isn't storage, it's temporal correctness. If I tell Claude in March that I'm using Postgres, and in June that I've moved to SQLite, a naive vector store now has two contradictory memories and will happily retrieve either one. The fix is typed edges with validity windows — memories that can supersede each other and know when they stopped being true. That's the next thing I'm building.

If you've solved contradiction resolution in a semantic memory store, I'd genuinely like to hear how.

It's live at https://mind-silo.com if you want to break it. I'll answer anything in the comments.