# dhruvtoprani/docflow-mcp architecture

> Architecture diagram of the GitHub repository dhruvtoprani/docflow-mcp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dhruvtoprani/docflow-mcp
- Repository: https://github.com/dhruvtoprani/docflow-mcp

## Overview

1. **What this repo is**
   - DocFlow MCP is a **TypeScript/Node.js MCP server** that turns a documentation URL plus a goal into a compact, implementation-oriented context pack.
   - It is **stateless**: no database, no auth layer, no user sessions, no persistence.

2. **Primary user-facing contract**
   - The main MCP tool is `extract_docs_context`.
   - Inputs: `url`, `goal`, and optional `stack`.
   - Output: structured documentation context, including a compact markdown pack, coverage signals, missing-info warnings, prompt-injection flags, and size-reduction stats.

3. **Core processing pipeline**
   - The extraction flow is concentrated under `src/core`.
   - The likely sequence is:
     - normalize and fetch the target page
     - parse and clean HTML
     - convert content to markdown
     - detect meaningful documentation sections
     - extract task-relevant context
     - compact implementation guidance
     - estimate token savings
     - sanitize suspicious prompt-injection content
   - This core layer is the main architecture boundary: it contains the repo’s business logic, independent of MCP transport.

4. **Document intelligence layer**
   - `detectDocSections.ts`, `extractDocsContext.ts`, and `generateContextPack.ts` are the central “understanding” modules.
   - `cleanHtml.ts`, `htmlToMarkdown.ts`, and `fetchPage.ts` handle the document ingestion and normalization side.
   - `compactImplementationContext.ts` and `estimateTokenSavings.ts` shape the final output for AI-assisted coding use.

5. **Safety and trust boundary**
   - `sanitizePromptInjection.ts` is a dedicated defense layer against hostile or misleading documentation text.
   - The README’s emphasis on hallucination reduction and injection detection suggests the server treats fetched docs as untrusted input and filters them before packaging.

6. **Schemas, types, and validation**
   - `src/schemas/extractDocsContext.schema.ts` defines the shape of the extraction output.
   - `src/types/docflow.ts` holds shared domain types.
   - This separates the free-form document processing from the contract exposed to MCP clients.

7. **Transport and server layer**
   - `src/server/mcpServer.ts` and `src/server/httpServer.ts` wrap the core extraction logic in MCP transports.
   - `src/server/tools/extractDocsContextTool.ts` is the bridge between MCP protocol requests and the core extractor.
   - The repo supports both **stdio MCP** and **Streamable HTTP MCP**, with `/mcp` as the HTTP endpoint and `/healthz` as the liveness check.

8. **HTTP / deployment surface**
   - `src/http.ts`, `src/index.ts`, and the `api/` folder indicate shared server entrypoints and deployment adapters.
   - `api/mcp.ts` and `api/healthz.ts` are likely serverless-compatible endpoints for hosted deployment.
   - `vercel.json` plus the README’s live URLs show the production target is Vercel-hosted HTTP MCP.

9. **Local developer workflow**
   - `src/dev/runExtract.ts` provides a local extraction runner for sample inputs.
   - `examples/` contains sample MCP config and sample input/output artifacts to demonstrate the tool contract and expected results.
   - `Dockerfile` supports containerized execution.

10. **Evaluation subsystem**
   - `src/eval/`, `eval/`, and the `npm run eval:*` scripts form a separate quality-checking pipeline.
   - This subsystem compares DocFlow’s extracted context against a rendered browser copy/paste baseline using rubric-based and fast keyword-based methods.
   - It is architecturally separate from the runtime server but important because it validates the extraction strategy.

11. **Tests as architecture documentation**
   - The `tests/` directory covers core extraction, prompt-injection sanitization, context pack generation, HTTP server behavior, and MCP harness integration.
   - The test suite reflects the main boundaries: core logic, transport layer, and end-to-end tool behavior.

12. **Static docs and landing page**
   - `docs/` and `public/` are separate static-site assets for the project landing page and connector instructions.
   - These assets are not part of the MCP runtime itself; they support discovery, documentation, and GitHub Pages deployment.

13. **Key architectural constraint**
   - The scope is intentionally narrow: **single-page documentation extraction** only.
   - The roadmap mentions future crawling and multi-page support, but the current repo is centered on one fetch-to-pack pipeline with minimal external dependencies.

14. **How to diagram it**
   - A clear diagram should show:
     - MCP client or HTTP caller
     - transport layer (`stdio` or `/mcp`)
     - tool wrapper
     - core document-processing pipeline
     - output schema/response
     - optional eval and static-docs side systems
   - The most important boundary is between **untrusted remote documentation content** and the **trusted structured output** produced by the core extractor.

Main entry points: [`src/server/tools/extractDocsContextTool.ts`](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/server/tools/extractDocsContextTool.ts) (Doc tool), [`src/core/extractDocsContext.ts`](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/extractDocsContext.ts) (Context extractor), [`src/core/generateContextPack.ts`](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/generateContextPack.ts) (Context pack), [`src/server/mcpServer.ts`](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/server/mcpServer.ts) (MCP server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_src_index["CLI entry<br/>entrypoint<br/>[index.ts]"]
  node_src_http["HTTP entry<br/>entrypoint<br/>[http.ts]"]
  node_mcp_server["MCP server<br/>stdio/http transport<br/>[mcpServer.ts]"]
  node_http_server["HTTP server<br/>streamable MCP<br/>[httpServer.ts]"]
  node_tool["Doc tool<br/>mcp tool"]
end

subgraph group_core["Core"]
  node_extractor["Context extractor<br/>core orchestrator"]
  node_normalize["Normalize URL<br/>url normalization<br/>[normalizeUrl.ts]"]
  node_fetch["Fetch page<br/>[fetchPage.ts]"]
  node_clean_html["Clean HTML<br/>html cleanup<br/>[cleanHtml.ts]"]
  node_html_md["To markdown<br/>html transform<br/>[htmlToMarkdown.ts]"]
  node_sections["Doc sections<br/>section detection"]
  node_sanitize["Sanitize text<br/>safety filter"]
  node_pack["Context pack<br/>packaging"]
  node_compact["Compact context<br/>compression"]
  node_savings["Token savings<br/>metrics"]
  node_schema["Output schema<br/>validation schema"]
  node_types["Docflow types<br/>shared types<br/>[docflow.ts]"]
end

subgraph group_quality["Quality"]
  node_eval_ab["AB eval<br/>evaluation<br/>[runAbEval.ts]"]
  node_eval_workflow["Workflow eval<br/>evaluation<br/>[runWorkflowEval.ts]"]
end

subgraph group_docs["Docs"]
  node_examples["Examples<br/>usage assets"]
  node_landing["Docs site<br/>static docs<br/>[index.html]"]
end

node_src_index -->|"starts"| node_mcp_server
node_src_http -->|"starts"| node_http_server
node_mcp_server -->|"registers"| node_tool
node_http_server -->|"routes to"| node_tool
node_tool -->|"calls"| node_extractor
node_extractor -->|"prepares url"| node_normalize
node_extractor -->|"loads"| node_fetch
node_fetch -->|"passes content"| node_clean_html
node_clean_html -->|"transforms"| node_html_md
node_html_md -->|"analyzes"| node_sections
node_sections -->|"filters"| node_sanitize
node_sanitize -->|"feeds"| node_pack
node_pack -->|"compresses"| node_compact
node_compact -->|"measures"| node_savings
node_pack -->|"validates"| node_schema
node_pack -->|"uses"| node_types
node_tool -->|"returns"| node_schema
node_eval_ab -.->|"benchmarks"| node_extractor
node_eval_workflow -.->|"benchmarks"| node_extractor
node_examples -.->|"demonstrates"| node_tool
node_landing -.->|"documents"| node_tool

click node_src_index "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/index.ts"
click node_src_http "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/http.ts"
click node_mcp_server "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/server/mcpServer.ts"
click node_http_server "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/server/httpServer.ts"
click node_tool "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/server/tools/extractDocsContextTool.ts"
click node_extractor "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/extractDocsContext.ts"
click node_normalize "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/normalizeUrl.ts"
click node_fetch "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/fetchPage.ts"
click node_clean_html "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/cleanHtml.ts"
click node_html_md "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/htmlToMarkdown.ts"
click node_sections "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/detectDocSections.ts"
click node_sanitize "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/sanitizePromptInjection.ts"
click node_pack "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/generateContextPack.ts"
click node_compact "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/compactImplementationContext.ts"
click node_savings "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/estimateTokenSavings.ts"
click node_schema "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/schemas/extractDocsContext.schema.ts"
click node_types "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/types/docflow.ts"
click node_eval_ab "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/eval/runAbEval.ts"
click node_eval_workflow "https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/eval/runWorkflowEval.ts"
click node_examples "https://github.com/dhruvtoprani/docflow-mcp/blob/main/examples/mcp-servers.example.json"
click node_landing "https://github.com/dhruvtoprani/docflow-mcp/blob/main/docs/index.html"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_src_index,node_src_http,node_mcp_server,node_http_server,node_tool toneBlue
class node_extractor,node_normalize,node_fetch,node_clean_html,node_html_md,node_sections,node_sanitize,node_pack,node_compact,node_savings,node_schema,node_types toneAmber
class node_eval_ab,node_eval_workflow toneMint
class node_examples,node_landing toneRose
```

## Components

### Runtime

MCP server and HTTP entrypoints

- [CLI entry](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/index.ts): entrypoint · Main process bootstrap and mode selection · `src/index.ts`
- [HTTP entry](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/http.ts): entrypoint · Shared HTTP server wiring · `src/http.ts`
- [MCP server](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/server/mcpServer.ts): stdio/http transport · Runs the MCP runtime over stdio or HTTP · `src/server/mcpServer.ts`
- [HTTP server](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/server/httpServer.ts): streamable MCP · HTTP transport with liveness and MCP endpoint support · `src/server/httpServer.ts`
- [Doc tool](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/server/tools/extractDocsContextTool.ts): mcp tool · Bridges MCP requests to the extractor · `src/server/tools/extractDocsContextTool.ts`

### Core

Document ingestion, understanding, and packing pipeline

- [Context extractor](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/extractDocsContext.ts): core orchestrator · Coordinates fetch, parsing, sectioning, compaction, and safety · `src/core/extractDocsContext.ts`
- [Normalize URL](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/normalizeUrl.ts): url normalization · Cleans and prepares the target docs URL · `src/core/normalizeUrl.ts`
- [Fetch page](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/fetchPage.ts): fetch · Loads the remote documentation page · `src/core/fetchPage.ts`
- [Clean HTML](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/cleanHtml.ts): html cleanup · Removes noise and prepares markup for parsing · `src/core/cleanHtml.ts`
- [To markdown](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/htmlToMarkdown.ts): html transform · Converts cleaned HTML into markdown text · `src/core/htmlToMarkdown.ts`
- [Doc sections](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/detectDocSections.ts): section detection · Finds meaningful documentation structure · `src/core/detectDocSections.ts`
- [Sanitize text](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/sanitizePromptInjection.ts): safety filter · Flags or removes prompt-injection style content · `src/core/sanitizePromptInjection.ts`
- [Context pack](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/generateContextPack.ts): packaging · Builds compact implementation-oriented output · `src/core/generateContextPack.ts`
- [Compact context](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/compactImplementationContext.ts): compression · Shrinks guidance for coding use · `src/core/compactImplementationContext.ts`
- [Token savings](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/core/estimateTokenSavings.ts): metrics · Estimates compression and size reduction · `src/core/estimateTokenSavings.ts`
- [Output schema](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/schemas/extractDocsContext.schema.ts): validation schema · Defines the structured extraction contract · `src/schemas/extractDocsContext.schema.ts`
- [Docflow types](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/types/docflow.ts): shared types · Common domain types for extraction results · `src/types/docflow.ts`

### Quality

Evaluation and test coverage

- [AB eval](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/eval/runAbEval.ts): evaluation · Compares extraction quality against baseline · `src/eval/runAbEval.ts`
- [Workflow eval](https://github.com/dhruvtoprani/docflow-mcp/blob/main/src/eval/runWorkflowEval.ts): evaluation · Runs workflow-oriented quality checks · `src/eval/runWorkflowEval.ts`

### Docs

Static site and usage assets

- [Examples](https://github.com/dhruvtoprani/docflow-mcp/blob/main/examples/mcp-servers.example.json): usage assets · Sample configs and input/output artifacts · `examples/mcp-servers.example.json`
- [Docs site](https://github.com/dhruvtoprani/docflow-mcp/blob/main/docs/index.html): static docs · Project landing and connector instructions · `docs/index.html`

## Connections

- CLI entry → MCP server: starts · Bootstraps the MCP runtime
- HTTP entry → HTTP server: starts · Bootstraps the HTTP server
- MCP server → Doc tool: registers · Exposes the extraction tool to MCP clients
- HTTP server → Doc tool: routes to · Forwards HTTP MCP requests to the tool
- Doc tool → Context extractor: calls · Invokes the core extraction pipeline
- Context extractor → Normalize URL: prepares url · Normalizes the requested documentation URL
- Context extractor → Fetch page: loads · Fetches remote documentation content
- Fetch page → Clean HTML: passes content · Sends fetched HTML into cleanup
- Clean HTML → To markdown: transforms · Converts cleaned markup to markdown
- To markdown → Doc sections: analyzes · Detects document structure and sections
- Doc sections → Sanitize text: filters · Removes suspicious or misleading text
- Sanitize text → Context pack: feeds · Provides trusted document content to packaging
- Context pack → Compact context: compresses · Creates a compact implementation context
- Compact context → Token savings: measures · Calculates token and size reduction
- Context pack → Output schema: validates · Shapes output to the response contract
- Context pack → Docflow types: uses · Shares domain types for the result payload
- Doc tool → Output schema: returns · Emits structured extraction output
- AB eval → Context extractor: benchmarks · Evaluates extractor quality against baseline
- Workflow eval → Context extractor: benchmarks · Evaluates workflow extraction behavior
- Examples → Doc tool: demonstrates · Shows expected MCP usage and sample outputs
- Docs site → Doc tool: documents · Explains connector and runtime usage
