# liustack/modsearch architecture

> Architecture diagram of the GitHub repository liustack/modsearch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/liustack/modsearch
- Repository: https://github.com/liustack/modsearch

## Overview

ModSearch adds web search, X search, and page fetching to coding-agent harnesses. A request enters through the CLI or the documented dsh plugin, is planned into engine roles and fallback order, then runs against configured providers; results and failures are returned as structured evidence. Configuration, key rotation, and quota cooldowns support that path, while local fetching adds HTML extraction and SSRF protections. The graph follows shown imports and calls plus README-documented capabilities; unsampled subsystem links are omitted where their wiring is not evidenced.

Main entry points: [`src/providers/index.ts`](https://github.com/liustack/modsearch/blob/main/src/providers/index.ts) (Engine registry), [`src/search.ts`](https://github.com/liustack/modsearch/blob/main/src/search.ts) (Search execution), [`src/config.ts`](https://github.com/liustack/modsearch/blob/main/src/config.ts) (Config and keys), [`src/main.ts`](https://github.com/liustack/modsearch/blob/main/src/main.ts) (ModSearch CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Agent entrypoints"]
  node_cli["ModSearch CLI<br/>[main.ts]"]
  node_dsh["dsh plugin<br/>[index.js]"]
end

subgraph group_orchestration["Request orchestration"]
  node_router["Role and fallback planner<br/>[router.ts]"]
  node_search["Search execution<br/>[search.ts]"]
  node_schema["Output schema<br/>[schema.ts]"]
  node_subprocess["CLI engine runner<br/>[subprocess.ts]"]
end

subgraph group_engines["Search and fetch engines"]
  node_registry["Engine registry<br/>[index.ts]"]
  node_firecrawl["Firecrawl provider<br/>[firecrawl.ts]"]
  node_keenable["Keenable provider<br/>[keenable.ts]"]
  node_youcom["You.com provider<br/>[youcom.ts]"]
  node_antigravity["Antigravity provider<br/>[antigravity.ts]"]
  node_tavily["Tavily provider<br/>[tavily.ts]"]
  node_exa["Exa provider<br/>[exa.ts]"]
  node_grok["Grok X provider<br/>[grok.ts]"]
  node_localfetch["Local page fetch<br/>[httpFetch.ts]"]
end

subgraph group_support["Configuration and safeguards"]
  node_config["Config and keys<br/>[config.ts]"]
  node_doctor["Health diagnostics<br/>[doctor.ts]"]
  node_cooldown[("Quota cooldown<br/>[cooldown.ts]")]
  node_network["SSRF network guards<br/>[network.ts]"]
  node_extract["HTML text extraction<br/>[htmlExtract.ts]"]
  node_system["Runtime environment<br/>[system.ts]"]
  node_endpoint["Endpoint settings<br/>[endpoint.ts]"]
  node_limits["Provider limits<br/>[limits.ts]"]
  node_api_keys["API key parsing<br/>[apiKeys.ts]"]
  node_proxy["Proxy resolution<br/>[proxy.ts]"]
  node_timed["Timed HTTP requests<br/>[timedRequest.ts]"]
  node_redact["Secret redaction<br/>[redact.ts]"]
end

node_agent(("Agent user"))

node_agent -->|"requests search"| node_cli
node_agent -.->|"uses plugin"| node_dsh
node_dsh -.->|"runs engine chain"| node_search
node_cli -->|"runs request"| node_search
node_cli -->|"loads settings"| node_config
node_cli -->|"runs diagnosis"| node_doctor
node_cli -->|"lists engines"| node_registry
node_doctor -->|"checks settings"| node_config
node_doctor -->|"checks cooldowns"| node_cooldown
node_doctor -->|"checks providers"| node_registry
node_search -->|"loads settings"| node_config
node_search -->|"plans run"| node_router
node_search -->|"selects engines"| node_registry
node_search -->|"reads cooldown"| node_cooldown
node_search -->|"runs commands"| node_subprocess
node_router -->|"reads engine settings"| node_config
node_router -->|"reorders cooling engines"| node_cooldown
node_router -->|"resolves providers"| node_registry
node_registry -->|"registers provider"| node_firecrawl
node_registry -->|"registers provider"| node_keenable
node_registry -->|"registers provider"| node_youcom
node_registry -->|"registers provider"| node_antigravity
node_registry -->|"registers provider"| node_tavily
node_registry -->|"registers provider"| node_exa
node_registry -->|"registers provider"| node_grok
node_registry -->|"registers provider"| node_localfetch
node_localfetch -->|"validates targets"| node_network
node_localfetch -->|"extracts visible text"| node_extract
node_localfetch -->|"resolves proxy"| node_proxy
node_firecrawl -->|"resolves endpoint"| node_endpoint
node_firecrawl -->|"applies limits"| node_limits
node_exa -->|"resolves endpoint"| node_endpoint
node_tavily -->|"resolves endpoint"| node_endpoint
node_firecrawl -->|"sends timed request"| node_timed
node_exa -->|"sends timed request"| node_timed
node_tavily -->|"sends timed request"| node_timed
node_timed -->|"resolves proxy"| node_proxy
node_timed -->|"scrubs errors"| node_redact
node_search -->|"splits keys"| node_api_keys
node_search -->|"scrubs failures"| node_redact
node_config -->|"parses key values"| node_api_keys
node_config -->|"masks credentials"| node_redact
node_cooldown -->|"reads cooldown policy"| node_config
node_cooldown -->|"scrubs reasons"| node_redact
node_search -.->|"formats result"| node_schema
node_antigravity -->|"uses runtime context"| node_system
node_grok -->|"uses runtime context"| node_system

click node_cli "https://github.com/liustack/modsearch/blob/main/src/main.ts"
click node_dsh "https://github.com/liustack/modsearch/blob/main/dsh/index.js"
click node_config "https://github.com/liustack/modsearch/blob/main/src/config.ts"
click node_doctor "https://github.com/liustack/modsearch/blob/main/src/doctor.ts"
click node_cooldown "https://github.com/liustack/modsearch/blob/main/src/cooldown.ts"
click node_router "https://github.com/liustack/modsearch/blob/main/src/router.ts"
click node_search "https://github.com/liustack/modsearch/blob/main/src/search.ts"
click node_schema "https://github.com/liustack/modsearch/blob/main/src/schema.ts"
click node_registry "https://github.com/liustack/modsearch/blob/main/src/providers/index.ts"
click node_firecrawl "https://github.com/liustack/modsearch/blob/main/src/providers/firecrawl.ts"
click node_keenable "https://github.com/liustack/modsearch/blob/main/src/providers/keenable.ts"
click node_youcom "https://github.com/liustack/modsearch/blob/main/src/providers/youcom.ts"
click node_antigravity "https://github.com/liustack/modsearch/blob/main/src/providers/antigravity.ts"
click node_tavily "https://github.com/liustack/modsearch/blob/main/src/providers/tavily.ts"
click node_exa "https://github.com/liustack/modsearch/blob/main/src/providers/exa.ts"
click node_grok "https://github.com/liustack/modsearch/blob/main/src/providers/grok.ts"
click node_localfetch "https://github.com/liustack/modsearch/blob/main/src/providers/httpFetch.ts"
click node_network "https://github.com/liustack/modsearch/blob/main/src/providers/http/network.ts"
click node_extract "https://github.com/liustack/modsearch/blob/main/src/providers/http/htmlExtract.ts"
click node_subprocess "https://github.com/liustack/modsearch/blob/main/src/subprocess.ts"
click node_system "https://github.com/liustack/modsearch/blob/main/src/system.ts"
click node_endpoint "https://github.com/liustack/modsearch/blob/main/src/providers/endpoint.ts"
click node_limits "https://github.com/liustack/modsearch/blob/main/src/providers/limits.ts"
click node_api_keys "https://github.com/liustack/modsearch/blob/main/src/util/apiKeys.ts"
click node_proxy "https://github.com/liustack/modsearch/blob/main/src/util/proxy.ts"
click node_timed "https://github.com/liustack/modsearch/blob/main/src/util/timedRequest.ts"
click node_redact "https://github.com/liustack/modsearch/blob/main/src/util/redact.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_dsh,node_agent toneBlue
class node_router,node_search,node_schema,node_subprocess toneAmber
class node_registry,node_firecrawl,node_keenable,node_youcom,node_antigravity,node_tavily,node_exa,node_grok,node_localfetch toneMint
class node_config,node_doctor,node_cooldown,node_network,node_extract,node_system,node_endpoint,node_limits,node_api_keys,node_proxy,node_timed,node_redact toneRose
```

## Components

### Agent entrypoints

- [ModSearch CLI](https://github.com/liustack/modsearch/blob/main/src/main.ts): `src/main.ts`
- [dsh plugin](https://github.com/liustack/modsearch/blob/main/dsh/index.js): `dsh/index.js`

### Request orchestration

- [Role and fallback planner](https://github.com/liustack/modsearch/blob/main/src/router.ts): `src/router.ts`
- [Search execution](https://github.com/liustack/modsearch/blob/main/src/search.ts): `src/search.ts`
- [Output schema](https://github.com/liustack/modsearch/blob/main/src/schema.ts): `src/schema.ts`
- [CLI engine runner](https://github.com/liustack/modsearch/blob/main/src/subprocess.ts): `src/subprocess.ts`

### Search and fetch engines

- [Engine registry](https://github.com/liustack/modsearch/blob/main/src/providers/index.ts): `src/providers/index.ts`
- [Firecrawl provider](https://github.com/liustack/modsearch/blob/main/src/providers/firecrawl.ts): `src/providers/firecrawl.ts`
- [Keenable provider](https://github.com/liustack/modsearch/blob/main/src/providers/keenable.ts): `src/providers/keenable.ts`
- [You.com provider](https://github.com/liustack/modsearch/blob/main/src/providers/youcom.ts): `src/providers/youcom.ts`
- [Antigravity provider](https://github.com/liustack/modsearch/blob/main/src/providers/antigravity.ts): `src/providers/antigravity.ts`
- [Tavily provider](https://github.com/liustack/modsearch/blob/main/src/providers/tavily.ts): `src/providers/tavily.ts`
- [Exa provider](https://github.com/liustack/modsearch/blob/main/src/providers/exa.ts): `src/providers/exa.ts`
- [Grok X provider](https://github.com/liustack/modsearch/blob/main/src/providers/grok.ts): `src/providers/grok.ts`
- [Local page fetch](https://github.com/liustack/modsearch/blob/main/src/providers/httpFetch.ts): `src/providers/httpFetch.ts`

### Configuration and safeguards

- [Config and keys](https://github.com/liustack/modsearch/blob/main/src/config.ts): `src/config.ts`
- [Health diagnostics](https://github.com/liustack/modsearch/blob/main/src/doctor.ts): `src/doctor.ts`
- [Quota cooldown](https://github.com/liustack/modsearch/blob/main/src/cooldown.ts): `src/cooldown.ts`
- [SSRF network guards](https://github.com/liustack/modsearch/blob/main/src/providers/http/network.ts): `src/providers/http/network.ts`
- [HTML text extraction](https://github.com/liustack/modsearch/blob/main/src/providers/http/htmlExtract.ts): `src/providers/http/htmlExtract.ts`
- [Runtime environment](https://github.com/liustack/modsearch/blob/main/src/system.ts): `src/system.ts`
- [Endpoint settings](https://github.com/liustack/modsearch/blob/main/src/providers/endpoint.ts): `src/providers/endpoint.ts`
- [Provider limits](https://github.com/liustack/modsearch/blob/main/src/providers/limits.ts): `src/providers/limits.ts`
- [API key parsing](https://github.com/liustack/modsearch/blob/main/src/util/apiKeys.ts): `src/util/apiKeys.ts`
- [Proxy resolution](https://github.com/liustack/modsearch/blob/main/src/util/proxy.ts): `src/util/proxy.ts`
- [Timed HTTP requests](https://github.com/liustack/modsearch/blob/main/src/util/timedRequest.ts): `src/util/timedRequest.ts`
- [Secret redaction](https://github.com/liustack/modsearch/blob/main/src/util/redact.ts): `src/util/redact.ts`

### Other components

- **Agent user**

## Connections

- Agent user → ModSearch CLI: requests search (evidence: `src/main.ts`)
- Agent user → dsh plugin: uses plugin (evidence: `README.md`)
- dsh plugin → Search execution: runs engine chain (evidence: `README.md`)
- ModSearch CLI → Search execution: runs request (evidence: `src/main.ts`)
- ModSearch CLI → Config and keys: loads settings (evidence: `src/main.ts`)
- ModSearch CLI → Health diagnostics: runs diagnosis (evidence: `src/main.ts`)
- ModSearch CLI → Engine registry: lists engines (evidence: `src/main.ts`)
- Health diagnostics → Config and keys: checks settings (evidence: `src/doctor.ts`)
- Health diagnostics → Quota cooldown: checks cooldowns (evidence: `src/doctor.ts`)
- Health diagnostics → Engine registry: checks providers (evidence: `src/doctor.ts`)
- Search execution → Config and keys: loads settings (evidence: `src/search.ts`)
- Search execution → Role and fallback planner: plans run (evidence: `src/search.ts`)
- Search execution → Engine registry: selects engines (evidence: `src/search.ts`)
- Search execution → Quota cooldown: reads cooldown (evidence: `src/search.ts`)
- Search execution → CLI engine runner: runs commands (evidence: `src/search.ts`)
- Role and fallback planner → Config and keys: reads engine settings (evidence: `src/router.ts`)
- Role and fallback planner → Quota cooldown: reorders cooling engines (evidence: `src/router.ts`)
- Role and fallback planner → Engine registry: resolves providers (evidence: `src/router.ts`)
- Engine registry → Firecrawl provider: registers provider (evidence: `src/providers/index.ts`)
- Engine registry → Keenable provider: registers provider (evidence: `src/providers/index.ts`)
- Engine registry → You.com provider: registers provider (evidence: `src/providers/index.ts`)
- Engine registry → Antigravity provider: registers provider (evidence: `src/providers/index.ts`)
- Engine registry → Tavily provider: registers provider (evidence: `src/providers/index.ts`)
- Engine registry → Exa provider: registers provider (evidence: `src/providers/index.ts`)
- Engine registry → Grok X provider: registers provider (evidence: `src/providers/index.ts`)
- Engine registry → Local page fetch: registers provider (evidence: `src/providers/index.ts`)
- Local page fetch → SSRF network guards: validates targets (evidence: `src/providers/httpFetch.ts`)
- Local page fetch → HTML text extraction: extracts visible text (evidence: `src/providers/httpFetch.ts`)
- Local page fetch → Proxy resolution: resolves proxy (evidence: `src/providers/httpFetch.ts`)
- Firecrawl provider → Endpoint settings: resolves endpoint (evidence: `src/providers/firecrawl.ts`)
- Firecrawl provider → Provider limits: applies limits (evidence: `src/providers/firecrawl.ts`)
- Exa provider → Endpoint settings: resolves endpoint (evidence: `src/providers/exa.ts`)
- Tavily provider → Endpoint settings: resolves endpoint (evidence: `src/providers/tavily.ts`)
- Firecrawl provider → Timed HTTP requests: sends timed request (evidence: `src/providers/firecrawl.ts`)
- Exa provider → Timed HTTP requests: sends timed request (evidence: `src/providers/exa.ts`)
- Tavily provider → Timed HTTP requests: sends timed request (evidence: `src/providers/tavily.ts`)
- Timed HTTP requests → Proxy resolution: resolves proxy (evidence: `src/util/timedRequest.ts`)
- Timed HTTP requests → Secret redaction: scrubs errors (evidence: `src/util/timedRequest.ts`)
- Search execution → API key parsing: splits keys (evidence: `src/search.ts`)
- Search execution → Secret redaction: scrubs failures (evidence: `src/search.ts`)
- Config and keys → API key parsing: parses key values (evidence: `src/config.ts`)
- Config and keys → Secret redaction: masks credentials (evidence: `src/config.ts`)
- Quota cooldown → Config and keys: reads cooldown policy (evidence: `src/cooldown.ts`)
- Quota cooldown → Secret redaction: scrubs reasons (evidence: `src/cooldown.ts`)
- Search execution → Output schema: formats result
- Antigravity provider → Runtime environment: uses runtime context (evidence: `src/providers/antigravity.ts`)
- Grok X provider → Runtime environment: uses runtime context (evidence: `src/providers/grok.ts`)
