# opendatalab/mineru-document-explorer architecture

> Architecture diagram of the GitHub repository opendatalab/mineru-document-explorer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/opendatalab/mineru-document-explorer
- Repository: https://github.com/opendatalab/mineru-document-explorer

## Overview

MinerU Document Explorer is an agent-oriented document knowledge engine. Users add document collections, which are indexed for cross-collection retrieval; agents can also deep-read individual documents and compile or maintain a linked wiki. CLI and MCP are separate access surfaces. The graph preserves the documented capabilities and the substantial query-expansion fine-tuning modules. Several implementation files are unsampled, so their internal links are omitted unless supported by the README or SOURCE INDEX references.

Main entry points: [`src/store.ts`](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/store.ts) (Document Store), [`src/mcp/tools/wiki.ts`](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/tools/wiki.ts) (Wiki Tools), [`src/mcp/tools/index.ts`](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/tools/index.ts) (MCP Tool Suites), [`finetune/train.py`](https://github.com/opendatalab/mineru-document-explorer/blob/main/finetune/train.py) (Model Training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_access["Agent Access"]
  node_cli["CLI<br/>[qmd.ts]"]
  node_mcp["MCP Server<br/>[server.ts]"]
  node_mcp_tools["MCP Tool Suites<br/>[index.ts]"]
  node_sdk["SDK<br/>[index.ts]"]
end

subgraph group_documents["Document Processing"]
  node_collections["Collections<br/>[collections.ts]"]
  node_indexing["Indexing and Chunking<br/>[indexing.ts]"]
  node_backends["Format Backends<br/>[registry.ts]"]
  node_deep_read["Deep Read<br/>[document.ts]"]
  node_store["Document Store<br/>[store.ts]"]
  node_database[("SQLite Database<br/>[db.ts]")]
end

subgraph group_retrieval["Search and Retrieval"]
  node_query_parser["Query Parser<br/>[query-parser.ts]"]
  node_search["Search<br/>[search.ts]"]
  node_hybrid["Hybrid Retrieval<br/>[hybrid-search.ts]"]
  node_llm["Local LLMs<br/>[llm.ts]"]
end

subgraph group_knowledge["Wiki Knowledge"]
  node_wiki_tools["Wiki Tools<br/>[wiki.ts]"]
  node_wiki_index["Wiki Index Generation<br/>[index-gen.ts]"]
  node_wiki_lint["Wiki Link Analysis<br/>[lint.ts]"]
  node_wiki_log["Wiki Activity Log<br/>[log.ts]"]
end

subgraph group_tuning["Query Fine-tuning"]
  node_dataset_schema["Dataset Schema<br/>[schema.py]"]
  node_reward["Expansion Scoring<br/>[reward.py]"]
  node_training["Model Training<br/>[train.py]"]
end

node_agent(("Agent / User"))

node_agent -->|"uses"| node_cli
node_agent -->|"connects via MCP"| node_mcp
node_agent -->|"uses programmatically"| node_sdk
node_cli -->|"indexes and searches"| node_store
node_mcp -->|"registers tools"| node_mcp_tools
node_mcp_tools -->|"includes"| node_wiki_tools
node_mcp_tools -->|"includes"| node_deep_read
node_collections -->|"configures"| node_store
node_store -->|"opens and queries"| node_database
node_store -->|"uses"| node_indexing
node_store -->|"selects format backend"| node_backends
node_store -->|"uses"| node_search
node_store -->|"uses"| node_hybrid
node_search -->|"uses"| node_llm
node_hybrid -->|"uses"| node_llm
node_query_parser -->|"produces search queries"| node_store
node_wiki_tools -->|"generates index"| node_wiki_index
node_wiki_tools -->|"uses"| node_wiki_lint
node_wiki_tools -->|"records and reads log"| node_wiki_log
node_wiki_tools -->|"reads and writes documents"| node_store
node_dataset_schema -.->|"scores formatted examples"| node_reward
node_training -->|"uses scoring"| node_reward
node_training -.->|"uses schema"| node_dataset_schema

click node_cli "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/cli/qmd.ts"
click node_mcp "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/server.ts"
click node_mcp_tools "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/tools/index.ts"
click node_sdk "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/index.ts"
click node_collections "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/collections.ts"
click node_indexing "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/backends/indexing.ts"
click node_backends "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/backends/registry.ts"
click node_deep_read "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/tools/document.ts"
click node_store "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/store.ts"
click node_database "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/db.ts"
click node_query_parser "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/query-parser.ts"
click node_search "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/search.ts"
click node_hybrid "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/hybrid-search.ts"
click node_llm "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/llm.ts"
click node_wiki_tools "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/tools/wiki.ts"
click node_wiki_index "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/wiki/index-gen.ts"
click node_wiki_lint "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/wiki/lint.ts"
click node_wiki_log "https://github.com/opendatalab/mineru-document-explorer/blob/main/src/wiki/log.ts"
click node_dataset_schema "https://github.com/opendatalab/mineru-document-explorer/blob/main/finetune/dataset/schema.py"
click node_reward "https://github.com/opendatalab/mineru-document-explorer/blob/main/finetune/reward.py"
click node_training "https://github.com/opendatalab/mineru-document-explorer/blob/main/finetune/train.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_mcp,node_mcp_tools,node_sdk,node_agent toneBlue
class node_collections,node_indexing,node_backends,node_deep_read,node_store,node_database toneAmber
class node_query_parser,node_search,node_hybrid,node_llm toneMint
class node_wiki_tools,node_wiki_index,node_wiki_lint,node_wiki_log toneRose
class node_dataset_schema,node_reward,node_training toneIndigo
```

## Components

### Agent Access

- [CLI](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/cli/qmd.ts): `src/cli/qmd.ts`
- [MCP Server](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/server.ts): `src/mcp/server.ts`
- [MCP Tool Suites](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/tools/index.ts): `src/mcp/tools/index.ts`
- [SDK](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/index.ts): `src/index.ts`

### Document Processing

- [Collections](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/collections.ts): `src/collections.ts`
- [Indexing and Chunking](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/backends/indexing.ts): `src/backends/indexing.ts`
- [Format Backends](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/backends/registry.ts): `src/backends/registry.ts`
- [Deep Read](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/tools/document.ts): `src/mcp/tools/document.ts`
- [Document Store](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/store.ts): `src/store.ts`
- [SQLite Database](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/db.ts): `src/db.ts`

### Search and Retrieval

- [Query Parser](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/query-parser.ts): `src/query-parser.ts`
- [Search](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/search.ts): `src/search.ts`
- [Hybrid Retrieval](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/hybrid-search.ts): `src/hybrid-search.ts`
- [Local LLMs](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/llm.ts): `src/llm.ts`

### Wiki Knowledge

- [Wiki Tools](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/mcp/tools/wiki.ts): `src/mcp/tools/wiki.ts`
- [Wiki Index Generation](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/wiki/index-gen.ts): `src/wiki/index-gen.ts`
- [Wiki Link Analysis](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/wiki/lint.ts): `src/wiki/lint.ts`
- [Wiki Activity Log](https://github.com/opendatalab/mineru-document-explorer/blob/main/src/wiki/log.ts): `src/wiki/log.ts`

### Query Fine-tuning

- [Dataset Schema](https://github.com/opendatalab/mineru-document-explorer/blob/main/finetune/dataset/schema.py): `finetune/dataset/schema.py`
- [Expansion Scoring](https://github.com/opendatalab/mineru-document-explorer/blob/main/finetune/reward.py): `finetune/reward.py`
- [Model Training](https://github.com/opendatalab/mineru-document-explorer/blob/main/finetune/train.py): `finetune/train.py`

### Other components

- **Agent / User**

## Connections

- Agent / User → CLI: uses (evidence: `README.md`)
- Agent / User → MCP Server: connects via MCP (evidence: `README.md`)
- Agent / User → SDK: uses programmatically (evidence: `src/index.ts`)
- CLI → Document Store: indexes and searches (evidence: `src/cli/qmd.ts`)
- MCP Server → MCP Tool Suites: registers tools (evidence: `src/mcp/server.ts`)
- MCP Tool Suites → Wiki Tools: includes (evidence: `src/mcp/tools/index.ts`)
- MCP Tool Suites → Deep Read: includes (evidence: `src/mcp/tools/index.ts`)
- Collections → Document Store: configures (evidence: `src/collections.ts`)
- Document Store → SQLite Database: opens and queries (evidence: `src/store.ts`)
- Document Store → Indexing and Chunking: uses (evidence: `src/store.ts`)
- Document Store → Format Backends: selects format backend (evidence: `src/store.ts`)
- Document Store → Search: uses (evidence: `src/store.ts`)
- Document Store → Hybrid Retrieval: uses (evidence: `src/store.ts`)
- Search → Local LLMs: uses (evidence: `src/search.ts`)
- Hybrid Retrieval → Local LLMs: uses (evidence: `src/hybrid-search.ts`)
- Query Parser → Document Store: produces search queries (evidence: `src/query-parser.ts`)
- Wiki Tools → Wiki Index Generation: generates index (evidence: `src/mcp/tools/wiki.ts`)
- Wiki Tools → Wiki Link Analysis: uses (evidence: `src/mcp/tools/wiki.ts`)
- Wiki Tools → Wiki Activity Log: records and reads log (evidence: `src/mcp/tools/wiki.ts`)
- Wiki Tools → Document Store: reads and writes documents (evidence: `src/mcp/tools/wiki.ts`)
- Dataset Schema → Expansion Scoring: scores formatted examples
- Model Training → Expansion Scoring: uses scoring (evidence: `finetune/train.py`)
- Model Training → Dataset Schema: uses schema
