How unclecode/doc2talk works
Project type and purpose
• Doc2Talk is a Python-based command-line tool and library that provides an interactive code-documentation chat interface.
• It combines a knowledge-graph index of your code and markdown docs with LLM calls for context decisions and response generation.
File structure
• Top level:
• docs/ → project documentation and usage examples
• src/doc2talk/ → core package modules
• tests/ → unit tests
• pyproject.toml → build and dependency configuration
• Key modules under src/doc2talk:
• __main__.py & cli.py → CLI entrypoint and argument parsing
• core.py & doc2talk.py → Doc2Talk orchestration (indexing, chatting, session management)
• docgraph.py → Knowledge-graph construction, BM25 search, serialization
• models.py → LLM configuration and wrappers
• Caching directories (not in this repo but described in README):
• ~/.doctalk/repos → cloned GitHub repos
• ~/.doctalk/index → serialized knowledge graphs
• ~/.doctalk/sessions → saved chat sessions
README for additional insights
• Describes three main subsystems:
1. DocGraph knowledge-graph engine (indexing, BM25 search, serialization)
2. Contextual decision making (LLM for deciding when/how to update context)
3. Repository & index caching (file-system stores and lazy loading)
• Explains CLI and Python API usage, LLM configurations, and performance optimizations.
a. Main components
• User (Terminal)
• CLI Module
• Doc2Talk Core
• DocGraph Engine
• LLM Service(s)
• File-System Caches (Repos, Indexes, Sessions)
• External GitHub / Local Code + Docs
b. Relationships and data flow
1. User types a command in the terminal
2. CLI parses arguments and invokes Doc2Talk core
3. Core checks for cached repo/index; if missing, it:
• Clones or reads code/docs from GitHub or local path
• Calls DocGraph to build or load the knowledge graph
• Serializes graph into index cache
4. For each user query:
• Core sends “should update context?” to Context-Decision LLM
• If needed, DocGraph BM25-searches graph for relevant passages
• Core assembles context and calls Generation LLM for response
• Core writes/updates session file and streams response to user
c. Architectural patterns & design principles
• Modular layering: CLI → Core orchestration → DocGraph → LLM → Data caches
• Lazy initialization: index built on demand
• Separation of concerns: knowledge-graph logic isolated from LLM logic and I/O
• Cache-first approach: avoid repeated cloning or reindexing
d. Key technologies & libraries
• Python packaging (pyproject.toml)
• LLM interaction (openai or similar) via models.py
• BM25 search engine inside docgraph.py
• Serialization (optimized binary format, memory-mapped I/O)
Essential architectural elements
• Persistent caches’ lifecycle (read/write operations).
• Flow of data from code/docs → knowledge graph → LLM → user.