# sreenuraj/web-mcp architecture

> Architecture diagram of the GitHub repository sreenuraj/web-mcp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sreenuraj/web-mcp
- Repository: https://github.com/sreenuraj/web-mcp

## Overview

web-mcp is a local, stdio MCP server that lets an agent conduct bounded web research. The workflow starts with a declared research plan, then searches and fetches pages under session budgets and policy checks; fetched content is extracted, ranked and returned as untrusted passages. When limits or policies require consent, the user decides through the approval flow. The README documents these subsystem relationships; sampled source confirms the server-to-engine tool dispatch, provider fallback, and approval persistence. Other internal links are based on README architecture documentation, not unsampled implementation details.

Main entry points: [`src/web_mcp/core.py`](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/core.py) (Research engine), [`src/web_mcp/policy`](https://github.com/sreenuraj/web-mcp/tree/main/src/web_mcp/policy) (Policy checks), [`src/web_mcp/extract/chunk_rank.py`](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/extract/chunk_rank.py) (Passage ranking), [`src/web_mcp/approval/broker.py`](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/approval/broker.py) (Approval broker)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["MCP entry"]
  node_server["Tool server<br/>[server.py]"]
end

subgraph group_research["Research engine"]
  node_engine["Research engine<br/>[core.py]"]
  node_config["Configuration<br/>[config.py]"]
  node_textutil["Text utilities<br/>[textutil.py]"]
end

subgraph group_guardrails["Guardrails"]
  node_policy["Policy checks"]
  node_leakage["Leak detection<br/>[leakage.py]"]
  node_relevance["Relevance checks<br/>[relevance.py]"]
  node_budget["Rate limits<br/>[budget.py]"]
  node_domains["Domain rules<br/>[domains.py]"]
  node_injection["Injection defense<br/>[injection.py]"]
  node_approval["Approval broker<br/>[broker.py]"]
end

subgraph group_web["Web retrieval"]
  node_providers["Provider chain<br/>[chain.py]"]
  node_provider_api["Provider interface<br/>[base.py]"]
  node_search_backends["Search backends"]
  node_safe_http["Safe HTTP<br/>[safe_http.py]"]
  node_extract_html["HTML extraction<br/>[html.py]"]
  node_extract_pdf["PDF extraction<br/>[pdf.py]"]
  node_rank["Passage ranking<br/>[chunk_rank.py]"]
end

subgraph group_state["State and audit"]
  node_sessions["Sessions and budgets<br/>[session.py]"]
  node_storage[("SQLite state and cache<br/>[storage.py]")]
  node_audit["Audit log<br/>[audit.py]"]
end

node_agent(("MCP agent"))
node_user(("User"))
node_websites(("Public websites"))

node_agent -->|"invokes tools"| node_server
node_server -->|"dispatches calls"| node_engine
node_engine -->|"manages sessions"| node_sessions
node_engine -->|"checks requests"| node_policy
node_engine -->|"searches through"| node_providers
node_engine -->|"fetches through"| node_safe_http
node_engine -->|"extracts pages"| node_extract_html
node_engine -->|"extracts PDFs"| node_extract_pdf
node_engine -->|"selects passages"| node_rank
node_engine -->|"requests approval"| node_approval
node_engine -->|"records events"| node_audit
node_engine -->|"uses settings"| node_config
node_policy -->|"scans for leaks"| node_leakage
node_policy -->|"checks relevance"| node_relevance
node_policy -->|"checks limits"| node_budget
node_policy -->|"checks domains"| node_domains
node_policy -->|"checks content"| node_injection
node_approval -->|"elicits decision"| node_user
node_approval -->|"persists decisions"| node_storage
node_sessions -->|"persists sessions"| node_storage
node_providers -->|"uses interface"| node_provider_api
node_providers -->|"dispatches search"| node_search_backends
node_search_backends -.->|"queries search services"| node_websites
node_safe_http -->|"fetches pages"| node_websites
node_extract_html -->|"provides page text"| node_rank
node_extract_pdf -->|"provides document text"| node_rank
node_rank -->|"uses text helpers"| node_textutil

click node_server "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/server.py"
click node_engine "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/core.py"
click node_config "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/config.py"
click node_sessions "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/session.py"
click node_policy "https://github.com/sreenuraj/web-mcp/tree/main/src/web_mcp/policy"
click node_leakage "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/leakage.py"
click node_relevance "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/relevance.py"
click node_budget "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/budget.py"
click node_domains "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/domains.py"
click node_injection "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/injection.py"
click node_approval "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/approval/broker.py"
click node_providers "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/providers/chain.py"
click node_provider_api "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/providers/base.py"
click node_search_backends "https://github.com/sreenuraj/web-mcp/tree/main/src/web_mcp/providers"
click node_safe_http "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/net/safe_http.py"
click node_extract_html "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/extract/html.py"
click node_extract_pdf "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/extract/pdf.py"
click node_rank "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/extract/chunk_rank.py"
click node_textutil "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/textutil.py"
click node_storage "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/storage.py"
click node_audit "https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/audit.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_server,node_user toneBlue
class node_engine,node_config,node_textutil toneAmber
class node_policy,node_leakage,node_relevance,node_budget,node_domains,node_injection,node_approval toneMint
class node_providers,node_provider_api,node_search_backends,node_safe_http,node_extract_html,node_extract_pdf,node_rank toneRose
class node_sessions,node_storage,node_audit,node_agent,node_websites toneIndigo
```

## Components

### MCP entry

- [Tool server](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/server.py): `src/web_mcp/server.py`

### Research engine

- [Research engine](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/core.py): `src/web_mcp/core.py`
- [Configuration](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/config.py): `src/web_mcp/config.py`
- [Text utilities](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/textutil.py): `src/web_mcp/textutil.py`

### Guardrails

- [Policy checks](https://github.com/sreenuraj/web-mcp/tree/main/src/web_mcp/policy): `src/web_mcp/policy`
- [Leak detection](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/leakage.py): `src/web_mcp/policy/leakage.py`
- [Relevance checks](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/relevance.py): `src/web_mcp/policy/relevance.py`
- [Rate limits](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/budget.py): `src/web_mcp/policy/budget.py`
- [Domain rules](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/domains.py): `src/web_mcp/policy/domains.py`
- [Injection defense](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/policy/injection.py): `src/web_mcp/policy/injection.py`
- [Approval broker](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/approval/broker.py): `src/web_mcp/approval/broker.py`

### Web retrieval

- [Provider chain](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/providers/chain.py): `src/web_mcp/providers/chain.py`
- [Provider interface](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/providers/base.py): `src/web_mcp/providers/base.py`
- [Search backends](https://github.com/sreenuraj/web-mcp/tree/main/src/web_mcp/providers): `src/web_mcp/providers`
- [Safe HTTP](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/net/safe_http.py): `src/web_mcp/net/safe_http.py`
- [HTML extraction](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/extract/html.py): `src/web_mcp/extract/html.py`
- [PDF extraction](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/extract/pdf.py): `src/web_mcp/extract/pdf.py`
- [Passage ranking](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/extract/chunk_rank.py): `src/web_mcp/extract/chunk_rank.py`

### State and audit

- [Sessions and budgets](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/session.py): `src/web_mcp/session.py`
- [SQLite state and cache](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/storage.py): `src/web_mcp/storage.py`
- [Audit log](https://github.com/sreenuraj/web-mcp/blob/main/src/web_mcp/audit.py): `src/web_mcp/audit.py`

### Other components

- **MCP agent**
- **User**
- **Public websites**

## Connections

- MCP agent → Tool server: invokes tools (evidence: `README.md`)
- Tool server → Research engine: dispatches calls (evidence: `src/web_mcp/server.py`)
- Research engine → Sessions and budgets: manages sessions (evidence: `README.md`)
- Research engine → Policy checks: checks requests (evidence: `README.md`)
- Research engine → Provider chain: searches through (evidence: `README.md`)
- Research engine → Safe HTTP: fetches through (evidence: `README.md`)
- Research engine → HTML extraction: extracts pages (evidence: `README.md`)
- Research engine → PDF extraction: extracts PDFs (evidence: `README.md`)
- Research engine → Passage ranking: selects passages (evidence: `README.md`)
- Research engine → Approval broker: requests approval (evidence: `README.md`)
- Research engine → Audit log: records events (evidence: `README.md`)
- Research engine → Configuration: uses settings (evidence: `src/web_mcp/core.py`)
- Policy checks → Leak detection: scans for leaks (evidence: `README.md`)
- Policy checks → Relevance checks: checks relevance (evidence: `README.md`)
- Policy checks → Rate limits: checks limits (evidence: `README.md`)
- Policy checks → Domain rules: checks domains (evidence: `README.md`)
- Policy checks → Injection defense: checks content (evidence: `README.md`)
- Approval broker → User: elicits decision (evidence: `README.md`)
- Approval broker → SQLite state and cache: persists decisions (evidence: `src/web_mcp/approval/broker.py`)
- Sessions and budgets → SQLite state and cache: persists sessions (evidence: `README.md`)
- Provider chain → Provider interface: uses interface (evidence: `src/web_mcp/providers/chain.py`)
- Provider chain → Search backends: dispatches search (evidence: `src/web_mcp/providers/chain.py`)
- Search backends → Public websites: queries search services (evidence: `README.md`)
- Safe HTTP → Public websites: fetches pages (evidence: `README.md`)
- HTML extraction → Passage ranking: provides page text (evidence: `README.md`)
- PDF extraction → Passage ranking: provides document text (evidence: `README.md`)
- Passage ranking → Text utilities: uses text helpers (evidence: `src/web_mcp/extract/chunk_rank.py`)
