# zerosumquant/claude-conversation-extractor architecture

> Architecture diagram of the GitHub repository zerosumquant/claude-conversation-extractor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zerosumquant/claude-conversation-extractor
- Repository: https://github.com/zerosumquant/claude-conversation-extractor

## Overview

1. **What this repo is**
   - A local, offline Python tool that reads Claude Code conversation logs from the user’s machine and turns them into searchable/exportable artifacts.
   - There is no backend, no network API, and no database; the primary boundary is the local filesystem under `~/.claude/projects`.

2. **Primary data source**
   - The input is Claude Code’s undocumented `chat_*.jsonl` files stored in per-project directories.
   - The tool scans that directory tree, discovers conversation sessions, and treats those files as the source of truth for both export and search.

3. **Core extraction pipeline**
   - `src/extract_claude_logs.py` appears to be the central extraction module.
   - Its role is to locate logs, parse JSONL records, normalize conversation structure, and emit exports in human-readable formats such as Markdown, plus structured formats like JSON and HTML.
   - This module is the main “read filesystem → parse messages → render output” path.

4. **Search subsystem**
   - `src/search_conversations.py` provides search over the extracted conversation corpus.
   - The search flow is local and text-based: scan conversations, match query terms, return hits with context/previews, and optionally feed matches into extraction/export actions.
   - README also suggests case-insensitive full-text search and direct search from the CLI.

5. **Real-time search subsystem**
   - `src/realtime_search.py` is a distinct interactive search engine path that updates results as the user types.
   - The dedicated tests for threading and UI imply this module coordinates incremental filtering, event handling, and background work so the interface stays responsive while searching locally large log sets.
   - If optional semantic search is enabled, it would be an extension of this layer rather than a separate service.

6. **Interactive UI boundary**
   - `src/interactive_ui.py` is the menu-driven front end used by `claude-start`.
   - It wraps discovery, search, and export operations into a user-facing terminal interface with navigation, prompts, and real-time search integration.
   - The UI is orchestration-only; it does not own storage or parsing logic.

7. **CLI boundary**
   - `src/search_cli.py` and the README’s `claude-extract`, `claude-search`, and backward-compatible `claude-logs` commands indicate multiple command entry points over the same core library.
   - The CLI layer is thin and delegates to the core extraction/search modules, making it suitable for scripting and automation.
   - The architecture is therefore “shared engine, multiple front doors” rather than separate implementations.

8. **Output and export formats**
   - The repo supports at least Markdown by default, plus JSON and HTML exports.
   - Output is generated from the normalized conversation model, not directly from raw JSONL, so formatting concerns are separated from parsing concerns.
   - The “detailed” export mode broadens the rendered transcript to include tool calls, MCP responses, system messages, and other metadata.

9. **Runtime and dependency model**
   - The project is pure Python and cross-platform, targeting Python 3.8+.
   - Core functionality is designed to work without external services; optional capabilities may add dependencies such as spaCy, but those are additive rather than required for the main flow.
   - Packaging is handled via `setup.py` and `pyproject.toml`, which likely expose the command-line entry points used in the README.

10. **Validation and quality gates**
   - The `tests/` tree shows strong coverage of extraction, search, UI, threading, integration, and error handling.
   - This mirrors the architecture: each major subsystem has dedicated tests, and there are integration tests that validate the seams between filesystem scanning, parsing, search, and UI behavior.
   - CI workflows in `.github/workflows/` run tests and publishing, so the release boundary is standard Python packaging rather than a deployed service.

11. **Documentation as an architectural aid**
   - `docs/user` and `docs/technical` document usage, install, changelog, and SEO/technical notes, but they do not add runtime components.
   - Their main architectural value is clarifying the intended user flows: discover logs, search them, then export selected sessions or all sessions.

12. **How the main flow fits together**
   - The practical data flow is: local Claude JSONL files → discovery/scanning → parsing/normalization → search index or filtered result set → rendering into Markdown/JSON/HTML → file output.
   - The interactive UI and CLI both sit on top of that same pipeline, so the architecture is best drawn as one core local processing engine with two presentation layers.

Main entry points: [`src/extract_claude_logs.py`](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py) (Extractor), [`src/interactive_ui.py`](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/interactive_ui.py) (Terminal UI), [`src/search_conversations.py`](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/search_conversations.py) (Search core), [`.github/workflows/test.yml`](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/.github/workflows/test.yml) (CI workflows)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_core["Core engine"]
  node_node_fs_source[("Claude logs<br/>filesystem source")]
  node_node_discovery["Discovery<br/>scan"]
  node_node_parse["Parse JSONL<br/>normalizer"]
  node_node_render["Render exports<br/>formatter"]
  node_node_extract_module{{"Extractor<br/>pipeline"}}
  node_node_search_core["Search core<br/>text search"]
  node_node_realtime_engine["Realtime engine<br/>interactive search<br/>[realtime_search.py]"]
end

subgraph group_group_interfaces["Interfaces"]
  node_node_cli_search["Search CLI<br/>cli entrypoint<br/>[search_cli.py]"]
  node_node_ui["Terminal UI<br/>interactive front end<br/>[interactive_ui.py]"]
end

subgraph group_group_quality["Quality and release"]
  node_node_setup["Packaging<br/>python package<br/>[setup.py]"]
  node_node_pyproject["Build config<br/>[pyproject.toml]"]
  node_node_requirements["Dependencies<br/>dependency manifests<br/>[base.txt]"]
  node_node_tests_extract["Extractor tests<br/>test suite<br/>[test_extractor.py]"]
  node_node_tests_search["Search tests<br/>test suite<br/>[test_search.py]"]
  node_node_tests_realtime["Realtime tests<br/>test suite"]
  node_node_tests_ui["UI tests<br/>test suite"]
  node_node_ci["CI workflows<br/>automation<br/>[test.yml]"]
end

subgraph group_group_docs["Docs"]
  node_node_docs["User docs<br/>documentation<br/>[INSTALL.md]"]
end

node_node_fs_source -->|"scan"| node_node_discovery
node_node_discovery -->|"load"| node_node_parse
node_node_parse -->|"normalize"| node_node_render
node_node_parse -->|"index/search"| node_node_search_core
node_node_render -->|"serve"| node_node_extract_module
node_node_extract_module -->|"share corpus"| node_node_search_core
node_node_search_core -->|"interactive mode"| node_node_realtime_engine
node_node_cli_search -->|"delegate"| node_node_search_core
node_node_cli_search -->|"delegate"| node_node_extract_module
node_node_ui -->|"embed"| node_node_realtime_engine
node_node_ui -->|"orchestrate"| node_node_extract_module
node_node_setup -->|"expose"| node_node_cli_search
node_node_setup -->|"expose"| node_node_ui
node_node_pyproject -->|"configure"| node_node_setup
node_node_requirements -->|"depends on"| node_node_extract_module
node_node_tests_extract -.->|"validate"| node_node_extract_module
node_node_tests_search -.->|"validate"| node_node_search_core
node_node_tests_realtime -.->|"validate"| node_node_realtime_engine
node_node_tests_ui -.->|"validate"| node_node_ui
node_node_ci -->|"run"| node_node_tests_extract
node_node_ci -->|"run"| node_node_tests_search
node_node_ci -->|"run"| node_node_tests_realtime
node_node_ci -->|"run"| node_node_tests_ui
node_node_docs -.->|"describe"| node_node_ui
node_node_docs -.->|"describe"| node_node_cli_search

click node_node_discovery "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py"
click node_node_parse "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py"
click node_node_render "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py"
click node_node_extract_module "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py"
click node_node_search_core "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/search_conversations.py"
click node_node_realtime_engine "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/realtime_search.py"
click node_node_cli_search "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/search_cli.py"
click node_node_ui "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/interactive_ui.py"
click node_node_setup "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/setup.py"
click node_node_pyproject "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/pyproject.toml"
click node_node_requirements "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/requirements/base.txt"
click node_node_tests_extract "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/tests/test_extractor.py"
click node_node_tests_search "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/tests/test_search.py"
click node_node_tests_realtime "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/tests/test_realtime_search_unit.py"
click node_node_tests_ui "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/tests/test_interactive_ui.py"
click node_node_ci "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/.github/workflows/test.yml"
click node_node_docs "https://github.com/zerosumquant/claude-conversation-extractor/blob/main/docs/user/INSTALL.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_fs_source,node_node_discovery,node_node_parse,node_node_render,node_node_extract_module,node_node_search_core,node_node_realtime_engine toneBlue
class node_node_cli_search,node_node_ui toneAmber
class node_node_setup,node_node_pyproject,node_node_requirements,node_node_tests_extract,node_node_tests_search,node_node_tests_realtime,node_node_tests_ui,node_node_ci toneMint
class node_node_docs toneRose
```

## Components

### Core engine

Local scanning, parsing, search, and rendering pipeline over Claude conversation logs.

- **Claude logs**: filesystem source · Undocumented chat_*.jsonl conversation files under ~/.claude/projects.
- [Discovery](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py): scan · Finds project directories and locates conversation sessions on disk. · `src/extract_claude_logs.py`
- [Parse JSONL](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py): normalizer · Reads raw log records and normalizes them into conversation structure. · `src/extract_claude_logs.py`
- [Render exports](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py): formatter · Formats normalized conversations into Markdown, JSON, HTML, and detailed transcript views. · `src/extract_claude_logs.py`
- [Extractor](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/extract_claude_logs.py): pipeline · Main read-parse-normalize-export pipeline for local conversation logs. · `src/extract_claude_logs.py`
- [Search core](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/search_conversations.py): text search · Searches extracted conversations with local text matching and context previews. · `src/search_conversations.py`
- [Realtime engine](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/realtime_search.py): interactive search · Incremental search loop that updates results as the user types. · `src/realtime_search.py`

### Interfaces

Command-line and terminal UI entry points that orchestrate the shared core.

- [Search CLI](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/search_cli.py): cli entrypoint · Thin command-line entry point for scripted search and extraction actions. · `src/search_cli.py`
- [Terminal UI](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/src/interactive_ui.py): interactive front end · Menu-driven terminal interface that orchestrates discovery, search, and export. · `src/interactive_ui.py`

### Quality and release

Tests, CI, and packaging that validate and ship the tool.

- [Packaging](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/setup.py): python package · Project metadata and console entry points for installation and distribution. · `setup.py`
- [Build config](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/pyproject.toml): build config · Project build and dependency configuration. · `pyproject.toml`
- [Dependencies](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/requirements/base.txt): dependency manifests · Base, dev, and optional dependency sets used across the tool. · `requirements/base.txt`
- [Extractor tests](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/tests/test_extractor.py): test suite · Coverage for extraction, alignment, comprehensive parsing, and error handling. · `tests/test_extractor.py`
- [Search tests](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/tests/test_search.py): test suite · Coverage for search behavior and search/extraction integration seams. · `tests/test_search.py`
- [Realtime tests](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/tests/test_realtime_search_unit.py): test suite · Coverage for threading, UI integration, and incremental search behavior. · `tests/test_realtime_search_unit.py`
- [UI tests](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/tests/test_interactive_ui.py): test suite · Coverage for the interactive terminal interface. · `tests/test_interactive_ui.py`
- [CI workflows](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/.github/workflows/test.yml): automation · Test and publish workflows in GitHub Actions. · `.github/workflows/test.yml`

### Docs

User and technical documentation that describe workflows and usage.

- [User docs](https://github.com/zerosumquant/claude-conversation-extractor/blob/main/docs/user/INSTALL.md): documentation · Install, changelog, and technical/user-facing guidance. · `docs/user/INSTALL.md`

## Connections

- Claude logs → Discovery: scan · Filesystem logs are discovered from the local Claude projects tree.
- Discovery → Parse JSONL: load · Discovered JSONL sessions are parsed into conversation records.
- Parse JSONL → Render exports: normalize · Parsed conversations are prepared for export rendering.
- Parse JSONL → Search core: index/search · Normalized conversations become the searchable corpus.
- Render exports → Extractor: serve · Rendering is part of the main extraction pipeline.
- Extractor → Search core: share corpus · The extractor and search path operate on the same conversation model.
- Search core → Realtime engine: interactive mode · Realtime search builds on the same local search logic.
- Search CLI → Search core: delegate · CLI commands call into shared search logic.
- Search CLI → Extractor: delegate · CLI commands can trigger extraction and export workflows.
- Terminal UI → Realtime engine: embed · The interactive UI incorporates realtime search for responsive filtering.
- Terminal UI → Extractor: orchestrate · The UI drives discovery and export actions without owning parsing logic.
- Packaging → Search CLI: expose · Packaging publishes command-line entry points.
- Packaging → Terminal UI: expose · Packaging exposes the terminal launcher.
- Build config → Packaging: configure · Build metadata and packaging settings support installation.
- Dependencies → Extractor: depends on · Core extraction relies on runtime dependencies declared in manifests.
- Extractor tests → Extractor: validate · Extraction tests exercise parsing and export behavior.
- Search tests → Search core: validate · Search tests verify matching, filtering, and integration behavior.
- Realtime tests → Realtime engine: validate · Realtime tests cover threading and incremental UI/search behavior.
- UI tests → Terminal UI: validate · UI tests verify menu-driven orchestration and prompts.
- CI workflows → Extractor tests: run · CI runs the extraction-focused test suite.
- CI workflows → Search tests: run · CI runs search validation.
- CI workflows → Realtime tests: run · CI runs realtime search validation.
- CI workflows → UI tests: run · CI runs interactive UI checks.
- User docs → Terminal UI: describe · Docs explain the interactive user flow.
- User docs → Search CLI: describe · Docs document command-line usage and workflows.
