# zhidian-cmd/deep_search architecture

> Architecture diagram of the GitHub repository zhidian-cmd/deep_search, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zhidian-cmd/deep_search
- Repository: https://github.com/zhidian-cmd/deep_search

## Overview

Deep Search MCP is a single-entry research pipeline: an MCP caller submits a query, the skill orchestrates search, page retrieval, quality/relevance filtering, deduplication, and a cited-source response with durable archives. Search aggregation is delegated to a Go CLI, while Python ranks and processes results. Separate branches support PDF preservation, cross-call deduplication, adaptive fetch-quality learning, and an engine/key GUI. Relationships follow the README’s documented workflow and sampled caller code; unsampled implementation details are not inferred.

Main entry points: [`skill.py`](https://github.com/zhidian-cmd/deep_search/blob/main/skill.py) (Pipeline orchestrator), [`web_fetch.py`](https://github.com/zhidian-cmd/deep_search/blob/main/web_fetch.py) (Fetch manager), [`search_engine/search.py`](https://github.com/zhidian-cmd/deep_search/blob/main/search_engine/search.py) (Search bridge), [`server.py`](https://github.com/zhidian-cmd/deep_search/blob/main/server.py) (MCP tool)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["MCP and orchestration"]
  node_server["MCP tool<br/>[server.py]"]
  node_skill["Pipeline orchestrator<br/>[skill.py]"]
  node_config["Runtime settings<br/>[config.py]"]
end

subgraph group_search["Search and ranking"]
  node_search_api["Search bridge<br/>[search.py]"]
  node_cli["Metasearch CLI"]
  node_providers(("Search providers"))
  node_rank["Result ranker<br/>[rank.py]"]
  node_gui["Engine key GUI<br/>[gui.py]"]
end

subgraph group_content["Fetch and quality"]
  node_fetch["Fetch manager<br/>[web_fetch.py]"]
  node_quality["Content filters<br/>[filters.py]"]
  node_filter_terms["Filter signals"]
  node_learning["Adaptive learning<br/>[filters_learn.py]"]
end

subgraph group_state["State and delivery"]
  node_ledger["Cross-round ledger<br/>[ledger.py]"]
  node_dedup["Content dedup<br/>[skill.py]"]
  node_pdf["PDF preservation<br/>[web_fetch.py]"]
  node_helpers["Text and URL helpers<br/>[helpers.py]"]
  node_archive["Markdown archive<br/>[config.py]"]
  node_response["Sources and answer<br/>[server.py]"]
end

node_user(("Research caller"))
node_http(("Web pages"))

node_user -->|"submits query"| node_server
node_server -->|"invokes"| node_skill
node_skill -->|"configures"| node_config
node_skill -->|"requests results"| node_search_api
node_search_api -->|"runs search"| node_cli
node_cli -->|"aggregates queries"| node_providers
node_search_api -->|"ranks results"| node_rank
node_gui -->|"runs test search"| node_search_api
node_skill -->|"fetches candidates"| node_fetch
node_fetch -->|"retrieves pages"| node_http
node_fetch -->|"scores content"| node_quality
node_quality -->|"uses signals"| node_filter_terms
node_fetch -->|"records outcomes"| node_learning
node_skill -->|"checks relevance"| node_quality
node_skill -->|"checks and records"| node_ledger
node_skill -->|"merges similar content"| node_dedup
node_fetch -->|"saves PDF files"| node_pdf
node_skill -.->|"formats archive text"| node_helpers
node_skill -->|"writes archive"| node_archive
node_skill -->|"assembles answer"| node_response
node_server -->|"returns result"| node_user

click node_server "https://github.com/zhidian-cmd/deep_search/blob/main/server.py"
click node_skill "https://github.com/zhidian-cmd/deep_search/blob/main/skill.py"
click node_config "https://github.com/zhidian-cmd/deep_search/blob/main/config.py"
click node_search_api "https://github.com/zhidian-cmd/deep_search/blob/main/search_engine/search.py"
click node_rank "https://github.com/zhidian-cmd/deep_search/blob/main/search_engine/ranking_meta/rank.py"
click node_gui "https://github.com/zhidian-cmd/deep_search/blob/main/search_engine/gui.py"
click node_fetch "https://github.com/zhidian-cmd/deep_search/blob/main/web_fetch.py"
click node_quality "https://github.com/zhidian-cmd/deep_search/blob/main/filters.py"
click node_filter_terms "https://github.com/zhidian-cmd/deep_search/blob/main/filters_keywords.py"
click node_learning "https://github.com/zhidian-cmd/deep_search/blob/main/filters_learn.py"
click node_ledger "https://github.com/zhidian-cmd/deep_search/blob/main/ledger.py"
click node_dedup "https://github.com/zhidian-cmd/deep_search/blob/main/skill.py"
click node_pdf "https://github.com/zhidian-cmd/deep_search/blob/main/web_fetch.py"
click node_helpers "https://github.com/zhidian-cmd/deep_search/blob/main/helpers.py"
click node_archive "https://github.com/zhidian-cmd/deep_search/blob/main/config.py"
click node_response "https://github.com/zhidian-cmd/deep_search/blob/main/server.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_server,node_skill,node_config toneBlue
class node_search_api,node_cli,node_providers,node_rank,node_gui toneAmber
class node_fetch,node_quality,node_filter_terms,node_learning toneMint
class node_ledger,node_dedup,node_pdf,node_helpers,node_archive,node_response toneRose
class node_user,node_http toneIndigo
```

## Components

### MCP and orchestration

- [MCP tool](https://github.com/zhidian-cmd/deep_search/blob/main/server.py): `server.py`
- [Pipeline orchestrator](https://github.com/zhidian-cmd/deep_search/blob/main/skill.py): `skill.py`
- [Runtime settings](https://github.com/zhidian-cmd/deep_search/blob/main/config.py): `config.py`

### Search and ranking

- [Search bridge](https://github.com/zhidian-cmd/deep_search/blob/main/search_engine/search.py): `search_engine/search.py`
- **Metasearch CLI**
- **Search providers**
- [Result ranker](https://github.com/zhidian-cmd/deep_search/blob/main/search_engine/ranking_meta/rank.py): `search_engine/ranking_meta/rank.py`
- [Engine key GUI](https://github.com/zhidian-cmd/deep_search/blob/main/search_engine/gui.py): `search_engine/gui.py`

### Fetch and quality

- [Fetch manager](https://github.com/zhidian-cmd/deep_search/blob/main/web_fetch.py): `web_fetch.py`
- [Content filters](https://github.com/zhidian-cmd/deep_search/blob/main/filters.py): `filters.py`
- [Filter signals](https://github.com/zhidian-cmd/deep_search/blob/main/filters_keywords.py): `filters_keywords.py`
- [Adaptive learning](https://github.com/zhidian-cmd/deep_search/blob/main/filters_learn.py): `filters_learn.py`

### State and delivery

- [Cross-round ledger](https://github.com/zhidian-cmd/deep_search/blob/main/ledger.py): `ledger.py`
- [Content dedup](https://github.com/zhidian-cmd/deep_search/blob/main/skill.py): `skill.py`
- [PDF preservation](https://github.com/zhidian-cmd/deep_search/blob/main/web_fetch.py): `web_fetch.py`
- [Text and URL helpers](https://github.com/zhidian-cmd/deep_search/blob/main/helpers.py): `helpers.py`
- [Markdown archive](https://github.com/zhidian-cmd/deep_search/blob/main/config.py): `config.py`
- [Sources and answer](https://github.com/zhidian-cmd/deep_search/blob/main/server.py): `server.py`

### Other components

- **Research caller**
- **Web pages**

## Connections

- Research caller → MCP tool: submits query (evidence: `README.md`)
- MCP tool → Pipeline orchestrator: invokes (evidence: `README.md`)
- Pipeline orchestrator → Runtime settings: configures (evidence: `skill.py`)
- Pipeline orchestrator → Search bridge: requests results (evidence: `README.md`)
- Search bridge → Metasearch CLI: runs search (evidence: `search_engine/search.py`)
- Metasearch CLI → Search providers: aggregates queries (evidence: `README.md`)
- Search bridge → Result ranker: ranks results (evidence: `search_engine/search.py`)
- Engine key GUI → Search bridge: runs test search (evidence: `search_engine/gui.py`)
- Pipeline orchestrator → Fetch manager: fetches candidates (evidence: `README.md`)
- Fetch manager → Web pages: retrieves pages (evidence: `README.md`)
- Fetch manager → Content filters: scores content (evidence: `README.md`)
- Content filters → Filter signals: uses signals (evidence: `filters.py`)
- Fetch manager → Adaptive learning: records outcomes (evidence: `web_fetch.py`)
- Pipeline orchestrator → Content filters: checks relevance (evidence: `skill.py`)
- Pipeline orchestrator → Cross-round ledger: checks and records (evidence: `README.md`)
- Pipeline orchestrator → Content dedup: merges similar content (evidence: `README.md`)
- Fetch manager → PDF preservation: saves PDF files (evidence: `README.md`)
- Pipeline orchestrator → Text and URL helpers: formats archive text
- Pipeline orchestrator → Markdown archive: writes archive (evidence: `README.md`)
- Pipeline orchestrator → Sources and answer: assembles answer (evidence: `README.md`)
- MCP tool → Research caller: returns result (evidence: `server.py`)