How benschneider/llm-lucid-memory works
Project Type and Purpose
• This is an open-source Python library plus CLI/GUI tool (“llm-lucid-memory”) designed to build a graph-based reasoning memory for LLMs. It chunks text or code, digests each chunk via LLM calls into structured “MemoryNode”s, links them into a graph, and provides retrieval (keyword/graph-traversal) to augment LLM prompts via a proxy server or direct pipeline.
File Structure Analysis
• Top-level:
• lucid_memory/ – core modules (chunker, digestor, processor, memory_graph, retriever, proxy_server, CLI & GUI controllers, managers)
• ui/ + streamlit_app.py – lightweight Tkinter & Streamlit interfaces
• examples/ – demo scripts for ingestion pipeline & client usage
• tests/ – pytest unit-test suite
• config & prompts: prompts.yaml, proxy_config.json
• Manager classes in lucid_memory/managers handle configuration, component wiring, and server lifecycle.
• Proxy server (FastAPI + uvicorn) sits between UI/CLI and the external LLM backend.
• Processor uses ThreadPoolExecutor for parallel digestion.
• MemoryGraph persists nodes (in memory_graph.json) and exposes traversal.
README Insights
• Purpose: overcome LLM context limits by structuring knowledge into chunks, digesting with multi-call LLM prompts, and linking into a reasoning graph.
• Supported modes: CLI (“lucid-memory”), Tkinter GUI, Streamlit app.
• Core flow: Chunk → Digest → MemoryNode creation → Graph linking → Retrieval → Proxy-augmented LLM calls → Chat UI.
• Tech stack: Python, FastAPI, uvicorn, PyYAML, requests, Tkinter, Streamlit.
• User Interfaces:
• CLI (lucid_memory/cli.py)
• Tkinter GUI (lucid_memory/gui.py + ui/)
• Streamlit App (streamlit_app.py)
• Controller & Managers:
• Controller (controller.py)
• ConfigManager, ComponentManager, ServerManager
• Ingestion Pipeline:
• Chunker (chunker.py)
• Digestor (digestor.py)
• Processor (processor.py)
• Memory Storage & Retrieval:
• MemoryNode (memory_node.py)
• MemoryGraph (memory_graph.py)
• Retriever (retriever.py)
• Proxy Layer:
• ProxyServer (proxy_server.py)
• ChainEngine (chain_engine.py) for assembling prompts & chains
• External Services:
• Local/Remote LLM API (e.g., Ollama, LMStudio, OpenAI-compatible)
1. User → CLI/GUI → Controller
2. Controller → ConfigManager loads proxy_config/prompts.yaml
3. Controller → Chunker splits input file → Processor parallelizes → Digestor invokes LLM → returns structured data
4. Structured data → MemoryNode → MemoryGraph stores and links nodes
5. User query (in chat) → ProxyServer receives → Retriever fetches relevant MemoryNode(s) via keyword & graph traversal
6. ProxyServer assembles context + user prompt via ChainEngine → forwards to LLM API → response → returns to UI
c. Architectural Patterns & Principles:
• Pipeline pattern for ingestion (chunker → digestor → processor).
• Manager pattern for configuration and service orchestration.
• Modular separation of concerns: ingestion, storage, retrieval, proxying, UI.
• Thread-pool parallelism in Processor for performance.
• Service-oriented: ProxyServer as a micro-service that wraps LLM calls.
d. Key Technologies & Libraries:
• FastAPI + uvicorn for proxy server
• PyYAML for prompt/config parsing
• AST module for Python code chunking
• ThreadPoolExecutor for concurrent digestion
• Tkinter & Streamlit for UI layers
• JSON storage for memory_graph.json
• Pytest for testing