How milla-jovovich/mempalace works
1. Repository purpose
MemPalace is a local-first Python memory system for AI conversations and project context. The core idea is to ingest raw files, turn them into searchable “drawers,” organize them into a palace taxonomy, and expose retrieval both through a CLI and an MCP server for agent integrations.
2. Top-level subsystems
The architecture splits into four main runtime paths:
- Ingestion and mining for project files and conversation exports
- Organization and retrieval via palace/room indexing and semantic search
- Structured memory via a temporal knowledge graph
- Integration surface through CLI, MCP tools, onboarding, and hooks
3. Ingestion pipeline
miner.py handles project-file mining, while convo_miner.py handles chat transcripts and chunks them into exchange-level units. normalize.py converts multiple chat formats into a common transcript shape, and split_mega_files.py pre-processes oversized export files into per-session pieces before mining. The data flow is: raw source files → normalized transcript/chunks → extracted memories and metadata → stored palace/graph records.
4. Palace model and retrieval boundary
palace_graph.py appears to manage the navigational structure described in the README: wings, rooms, halls, tunnels, and drawer locations. This is the primary architectural boundary for search and browsing. It lets the system narrow retrieval by domain before doing deeper semantic lookup, which is why wing/room filters materially improve recall.
5. Semantic search layer
searcher.py is the search entry point exposed in both the CLI and Python API. The README indicates ChromaDB-backed local vector search, so the retrieval layer likely combines embedded drawer content with wing/room metadata to support filtered semantic search. This is the main query path for “what did we decide about X?”-style questions.
6. Knowledge graph layer
knowledge_graph.py stores temporal entity-relationship triples with validity windows, current-state queries, historical queries, and invalidation. This is a separate structured-memory subsystem from vector search: it models facts, assignments, decisions, and timelines rather than free-text recall. The README explicitly describes SQLite as the local backing store, which makes this a durable, offline graph boundary.
7. AAAK compression and memory layers
dialect.py implements the AAAK shorthand described in the README, and layers.py defines the L0–L3 memory stack used for wake-up, critical facts, and deeper retrieval. These components are the compression and presentation boundary: they turn stored memories into compact context for models while preserving the underlying verbatim source in drawers.
8. Entity detection and registry
entity_detector.py and entity_registry.py support automatic identification and consistent labeling of people/projects/topics. These are the glue between unstructured text and the palace/graph model, helping route content into the correct wing, room, or entity code. general_extractor.py likely provides broader classification for general memories such as decisions, milestones, and preferences.
9. User-facing interfaces
cli.py and __main__.py provide the command-line entry point for init, mine, search, wake-up, compress, split, and status flows. mcp_server.py is the agent-facing integration boundary, exposing the repository’s functionality as MCP tools so Claude/Cursor/other compatible clients can call search, graph queries, and diary operations without manual CLI use.
10. Setup, configuration, and onboarding
config.py loads global config, palace path, and entity mappings; onboarding.py generates the initial palace bootstrap and wing configuration. The repository is designed so the first-run experience creates the local memory world, after which mining and retrieval are repeatable and offline.
11. Extensions and automation
The hooks/ directory contains Claude Code save/precompact hooks, which automate periodic memory capture and emergency saves before context truncation. examples/ demonstrates project mining, convo import, and MCP setup, showing how the system is meant to be embedded into existing workflows rather than used as a standalone app.
12. Validation and reproducibility
tests/ covers configuration, normalization, mining, and conversation ingestion, which are the highest-risk transformation boundaries in the system. benchmarks/ contains reproducible runners and methodology for LongMemEval, LoCoMo, and related evaluations, indicating that retrieval quality is a core architectural concern, not just a utility feature.
13. Operational profile
This is a pure local Python package with no mandatory cloud dependency. The runtime stack is centered on Python 3.9+, local storage/indexes, and offline retrieval, with external services only appearing as optional model clients or MCP consumers on the edge of the system. The main architectural promise is that raw data stays local while the system adds structure, compression, and searchability around it.
Main entry points: cli.py CLI · miner.py Miner · searcher.py Searcher · knowledge_graph.py Knowledge Graph
Components 18
Ingestion
Raw files and chat exports are normalized, mined, and split into memory units.
- Miner · Extracts memories from project files and source content.
mempalace/miner.py - Convo Miner · Processes conversation exports into chunked exchange units.
mempalace/convo_miner.py - Normalize · Converts multiple chat formats into a common transcript shape.
mempalace/normalize.py - Split Files · Breaks oversized conversation exports into per-session pieces.
mempalace/split_mega_files.py
Palace Retrieval
Palace taxonomy and semantic search for browsing and recall.
- Palace Graph · Manages wings, rooms, halls, tunnels, and drawer locations.
mempalace/palace_graph.py - Searcher · Entry point for filtered local retrieval over embedded drawers.
mempalace/searcher.py - Room Detector · Locates likely wing and room boundaries for mined content.
mempalace/room_detector_local.py
Knowledge Graph
Temporal structured memory for entities, facts, and history.
- Knowledge Graph · Stores entities, relations, validity windows, and historical state.
mempalace/knowledge_graph.py
Interfaces
CLI, MCP, onboarding, and automation entry points.
- Onboarding · Creates the initial palace and wing configuration.
mempalace/onboarding.py - CLI · Main command-line interface for init, mine, search, and status.
mempalace/cli.py - __main__ · Package entrypoint that dispatches to the CLI.
mempalace/__main__.py - MCP Server · Exposes search and memory operations to agent clients.
mempalace/mcp_server.py
Support
Configuration, compression, entity routing, and general extraction.
- AAAK Dialect · Encodes compact memory shorthand for model context.
mempalace/dialect.py - Memory Layers · Defines the L0-L3 memory stack for wake-up and deeper retrieval.
mempalace/layers.py - Entity Detector · Identifies people, projects, and topics in text.
mempalace/entity_detector.py - Entity Registry · Maintains consistent labels and codes for recognized entities.
mempalace/entity_registry.py - General Extractor · Classifies decisions, milestones, preferences, and other general memories.
mempalace/general_extractor.py - Config · Loads local paths, settings, and entity mappings.
mempalace/config.py
Connections 21
- Split Files → Normalize: prepares · Large exports are split before transcript normalization.
- Normalize → Convo Miner: feeds · Normalized transcripts are chunked into mineable conversation units.
- Convo Miner → Miner: supplies · Conversation-derived units enter the mining pipeline.
- Miner → Palace Graph: indexes · Mined memories are placed into the palace taxonomy.
- Miner → Knowledge Graph: writes facts · Structured entities and relations are stored in the temporal graph.
- Room Detector → Palace Graph: routes · Detected rooms and wings guide retrieval placement.
- Palace Graph → Searcher: filters · Palace metadata narrows semantic search scope.
- Searcher → Memory Layers: returns context · Search results are compressed into layered memory context.
- AAAK Dialect → Memory Layers: compresses · AAAK shorthand provides compact presentation of memories.
- Entity Detector → Entity Registry: normalizes · Detected entities are mapped to stable registry labels.
- Entity Registry → Knowledge Graph: anchors · Entity identities anchor graph facts over time.
- General Extractor → Miner: classifies · General memory types are extracted during mining.
- Config → Onboarding: configures · Bootstrap uses config to create the local memory world.
- Config → CLI: loads · CLI commands operate against the configured local store.
- Onboarding → CLI: bootstraps · Initial setup is initiated through the command surface.
- CLI → Miner: invokes · The CLI runs mining workflows.
- CLI → Searcher: queries · The CLI exposes retrieval entry points.
- CLI → Knowledge Graph: queries · The CLI can inspect structured memory state.
- __main__ → CLI: dispatches · Package execution forwards to the CLI.
- MCP Server → Searcher: exposes · Agent tools wrap semantic search.
- MCP Server → Knowledge Graph: exposes · Agent tools wrap graph queries and updates.