How sahenjp/rustxng works
RustXNG is a self-hosted metasearch service: a user submits a query through HTTP or MCP, the request is parsed and dispatched to configured external providers and/or selected local indexes, and results are merged and returned in the requested format. Separate capabilities include fetching readable page text, MCP page reading, autocomplete, plugins, and answerers. The graph reflects the sampled source and README; several subsystem links are documented rather than verified from implementation excerpts.
Main entry points: aggregator.rs Search Orchestrator · handlers.rs HTTP API and UI · direct.rs Page Fetching · mod.rs MCP Tools
Components 24
User Interfaces
- HTTP API and UI
src/web/handlers.rs - MCP Tools
src/mcp/mod.rs - Response Formats
src/web/feeds.rs
Search Orchestration
- Query Parsing
src/query.rs - Search Orchestrator
src/aggregator.rs - Merge and Ranking
src/results.rs - Search Plugins
src/plugins/mod.rs - Answerers
src/answerers/mod.rs - Autocomplete
src/autocomplete.rs - External Bangs
src/external_bangs.rs
Search Sources
- Engine Registry
src/engine/mod.rs - Provider Engines
src/engines/mod.rs - External Providers
- Local Index
src/localindex.rs - Local Crawling
src/crawl.rs - Wikipedia Index Import
src/bin/rustxng-wikidump.rs
Page Retrieval
- Page Fetching
src/fetch/direct.rs - Text Extraction
src/content.rs - Optional Rendering
src/fetch/render.rs
Runtime Services
- HTTP Networking
src/network.rs - Response Cache
src/cache.rs - Runtime Settings
src/settings.rs - Metrics
src/metrics.rs
Other components
- Search Client
Connections 28
- Search Client → HTTP API and UI: submits search
README.md - Search Client → MCP Tools: uses tools
README.md - HTTP API and UI → Query Parsing: parses query
src/web/handlers.rs - HTTP API and UI → Search Orchestrator: runs search
src/web/handlers.rs - MCP Tools → Search Orchestrator: runs search
src/mcp/mod.rs - MCP Tools → Page Fetching: reads pages
src/mcp/mod.rs - Query Parsing → Search Orchestrator: provides request
src/aggregator.rs - Search Orchestrator → Engine Registry: selects engines
src/aggregator.rs - Engine Registry → Provider Engines: builds engines
src/engine/mod.rs - Search Orchestrator → External Providers: queries providers
README.md - Search Orchestrator → Local Index: searches index
src/aggregator.rs - Local Crawling → Local Index: builds index
README.md - Wikipedia Index Import → Local Index: imports dump
README.md - Search Orchestrator → Merge and Ranking: merges results
src/aggregator.rs - Search Orchestrator → Search Plugins: runs plugins
src/aggregator.rs - Search Orchestrator → Answerers: adds answers
src/aggregator.rs - HTTP API and UI → Autocomplete: serves suggestions
src/web/handlers.rs - Query Parsing → External Bangs: resolves bangs
src/query.rs - HTTP API and UI → Response Formats: formats response
src/web/handlers.rs - HTTP API and UI → Page Fetching: fetches raw text
src/web/handlers.rs - Page Fetching → Text Extraction: extracts text
src/fetch/direct.rs - Page Fetching → Optional Rendering: optionally renders
src/fetch/direct.rs - Page Fetching → HTTP Networking: requests pages
src/fetch/direct.rs - Search Orchestrator → HTTP Networking: requests providers
src/aggregator.rs - HTTP Networking → Response Cache: reads and writes
src/network.rs - Runtime Settings → Engine Registry: configures engines
src/engine/mod.rs - Runtime Settings → Page Fetching: configures fetching
src/fetch/mod.rs - HTTP API and UI → Metrics: exposes metrics
README.md