How aiming-lab/simplemem works
SimpleMem is a lifelong memory system for LLM agents: conversations or multimodal inputs become structured memories, are indexed, and are retrieved to answer later queries. The repository also includes MCP access, cross-session memory, and EvolveMem’s retrieval-optimization capabilities. The README documents the high-level workflow and product branches; source-index references support several internal dependencies. Many subsystems are unsampled, so their detailed call paths are omitted rather than inferred.
Main entry points: app.py Multimodal API · router.py SimpleMem API · memory_builder.py Memory builder · hybrid_retriever.py Hybrid retrieval
Components 20
Text memory
- SimpleMem API
simplemem/router.py - Memory builder
simplemem/core/memory_builder.py - Vector store
simplemem/core/database/vector_store.py - Hybrid retrieval
simplemem/core/hybrid_retriever.py - Answer generation
simplemem/core/answer_generator.py - LLM client
simplemem/core/utils/llm_client.py
Multimodal memory
- Multimodal API
simplemem/multimodal/app.py - Media processors
simplemem/multimodal/processors - Multimodal storage
simplemem/multimodal/storage - Knowledge graph
simplemem/multimodal/knowledge - Multimodal retrieval
simplemem/multimodal/retrieval
Agent access
- MCP server
MCP/server/mcp_handler.py - HTTP server
MCP/server/http_server.py
Cross-session
- Session orchestrator
cross/orchestrator.py - Session storage
cross/storage_sqlite.py
Memory evolution
- Evolution manager
simplemem/evolver/manager.py - Evolution store
simplemem/evolver/store.py - Meta-experience
simplemem/multimodal/evolution/experience_engine.py
Other components
- LLM agent
- MCP client
Connections 17
- LLM agent → SimpleMem API: submits memory
README.md - SimpleMem API → Memory builder: stores dialogue
- Memory builder → Vector store: indexes memories
simplemem/core/memory_builder.py - SimpleMem API → Hybrid retrieval: requests recall
- Hybrid retrieval → Vector store: searches memories
simplemem/core/hybrid_retriever.py - SimpleMem API → Answer generation: generates answer
- Answer generation → LLM client: calls model
simplemem/core/answer_generator.py - LLM agent → Multimodal API: sends media
README.md - Multimodal API → Media processors: processes modalities
- Multimodal API → Multimodal storage: stores media memory
- Multimodal API → Knowledge graph: builds knowledge graph
- Multimodal API → Multimodal retrieval: retrieves memories
- MCP client → MCP server: invokes tools
README.md - MCP server → HTTP server: serves requests
- Session orchestrator → Session storage: persists sessions
- Evolution manager → Evolution store: reads and writes
simplemem/evolver/manager.py - Meta-experience → Meta-experience: records outcomes
simplemem/multimodal/evolution/experience_engine.py