How basicmachines-co/basic-memory works
Project Type and Purpose
• Basic Memory is a local knowledge base system that lets users and LLMs (such as Claude) interact by reading and writing structured Markdown files. It includes both an API layer (to support MCP integration for LLMs) and a CLI for command-based interactions. It manages a knowledge graph extracted from Markdown files and uses a SQLite database for indexing and searching.
• Its goal is to provide “persistent memory” for conversations, allowing for bi-directional updates between human-provided notes and AI assistants. It is a full-stack project built around local file storage, API/CLI communication, and integration with external AI services.
File Structure
– api: Implements HTTP endpoints (routers for knowledge, memory, search, and project info) to support the MCP protocol.
– cli: Contains command implementations allowing command-line interactions (e.g., sync, import, status, and project commands).
– mcp: Implements the Model Context Protocol (both client and server components) to integrate with LLMs like Claude.
– models, repository, schemas: The layered abstraction for data – defining entities, database repositories, and data transport formats.
– markdown: Logic to process Markdown files and extract structured data such as Observations and Relations.
– services: Business logic components including context service, entity service, file service, and search service.
• Other supporting files include configuration (pyproject.toml, Dockerfile, Makefile) and tests across multiple domains (API, CLI, markdown, repository, etc.).
a. Main Components:
• Input Layer:
– CLI Interface: Allows users to run commands like sync, import, and note manipulation.
– API Server: Exposes HTTP endpoints (via routers) for MCP-based communication (used by LLMs and integration with Claude Desktop).
• Business Logic Layer:
– MCP Component: Implements the Model Context Protocol to interact with external LLMs (bidirectional read/write for notes).
– Services: Contains logic for entity management, context building, and search operations.
• Data Layer:
– Repository & Database: Uses a SQLite database to index and search Markdown files and store metadata.
– File System Storage: Houses all the Markdown files that represent the persistent knowledge base.
• Supporting Components:
– Markdown Parser/Processor: Converts Markdown files into structured entities (knowledge, observations, relations).
– Data Sync Module: Manages watch processes and real-time synchronization between file changes and the database.
– Configuration/Environment: Build scripts (Makefile, Dockerfile) and configuration files (pyproject.toml, installer scripts) that help deploy and run the system.
b. Relationships and Interactions:
• The CLI and API Server both interact with the core “services” module.
• The MCP component (invoked via both API endpoints and CLI) communicates with external LLMs by adhering to the MCP protocol.
• Services access the repository layer, which in turn reads/writes to both the SQLite database and the local Markdown file system.
• The Markdown parser/processor extracts semantic entities from the files, enabling the repository to maintain and update the knowledge graph.
• Data flows should show:
– User commands (via CLI/API) trigger service calls.
– Services interact with the repository to update/read persistent storage.
– Data synchronization between file changes (Markdown) and SQLite occurs in near-real time.
c. Architectural Patterns & Design Principles:
• Separation of Concerns: Different layers (interface, business logic, data persistence) are clearly demarcated.
• Modular Design: Components are organized in distinct modules (api, cli, mcp, services, repository, markdown) with clear responsibilities.
• Local-First & Lightweight: Uses local files (Markdown) and a SQLite database, avoiding heavy cloud infrastructure.
• Extensibility: The use of routers and commands makes it adaptable to both CLI and HTTP/MCP driven interactions.
d. Relevant Technologies, Frameworks, and Tools:
• Python (with support for Python 3.12+)
• SQLite for persistent storage
• Docker (via Dockerfile) and Makefile for deployment and automation
• CI/CD and testing via GitHub workflows (in .github/workflows)
• External integrations with Claude Desktop and Smithery for MCP configuration
• Markdown processing libraries (implied in src/basic_memory/markdown)