# egoist/fetch-mcp architecture

> Architecture diagram of the GitHub repository egoist/fetch-mcp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/egoist/fetch-mcp
- Repository: https://github.com/egoist/fetch-mcp

## Overview

##### Project Type and Purpose
- The project is a command-line MCP server focused on fetching content. It primarily deals with two tasks:
- Fetching URLs (returning HTML or Markdown).
- Fetching YouTube video transcripts.
- The README indicates that it runs as either a stdio or SSE server, and the CLI is the entry point (via “npx -y fetch-mcp”).

##### File Structure
- The files are relatively flat, with key files including:
- cli.js: Likely the command-line interface module that handles execution and input parameters.
- index.ts: Probably the main entry point or application bootstrapping logic.
- markdown.ts: Likely responsible for handling or converting content to Markdown format.
- youtube.ts: Implements fetching YouTube transcripts.
- utils.ts: Contains common helper functions used by other modules.
- TypeScript configuration files (tsconfig.json, shims.d.ts) and package configuration files indicate a TypeScript project.
- There are no separate folders for frontend/backend, so the project is primarily a CLI/utility tool.

##### README for Additional Insights
- The README provides usage instructions for both standard input/output and SSE server modes.
- It explicitly lists “fetch_url” and “fetch_youtube_transcript” as the two core tools.
- The sponsorship by ChatWise suggests integration or compatibility with a larger ecosystem, though that aspect is handled externally.

a. Main Components:
- Command-Line Interface (CLI): Entry point for the application (cli.js).
- Application Core: Main processing unit (index.ts) that initializes and routes commands.
- URL Fetcher: Component that fetches URLs (logic likely split across index.ts, utils.ts, and markdown.ts).
- YouTube Transcript Fetcher: Dedicated module for retrieving YouTube transcripts (youtube.ts).
- Data Formatter/Parser: Optional but suggested by markdown.ts for converting or handling data in Markdown format.
- Server Modes: Represent the two operating modes (stdio and SSE) as separate system behaviors that might share common logic.

b. Relationships and Interactions:
- The CLI interfaces with the Application Core by parsing command-line arguments (including mode selection, e.g., --sse).
- The Application Core then directs the control flow:
- If the task is to fetch a URL, it calls into the URL Fetcher and Data Formatter to process and possibly convert HTML to Markdown.
- If the task involves YouTube, the core delegates to the YouTube Transcript Fetcher.
- Both fetchers may utilize shared utilities provided by the utils.ts module.
- Data flows from these components back to the CLI output or is pushed through SSE if that mode is activated.

c. Architectural Patterns and Design Principles:
- Separation of Concerns: Each module is responsible for a specific task (CLI, URL fetching, YouTube transcript fetching).
- Modular Design: Encourages extensions or modifications in individual components without affecting the entire system.
- Clear input-processing-output flow, with routing in the main index.ts.
- Optional Server Mode Pattern: Handling different modes (stdio vs. SSE) can be seen as a configuration-based branching within the architecture.

d. Relevant Technologies, Frameworks, or Libraries:
- TypeScript as evidenced by .ts files and tsconfig.json.
- Bun (bun.lock) might be used for package management or running the code.
- Node.js runtime (implied by CLI usage and JavaScript/TypeScript nature).

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["CLI Handler"]:::cli
    B["Application Core/Router"]:::core
    C{"Task Type"}:::decision
    D["URL Fetcher & Data Formatter"]:::fetcher
    E["YouTube Transcript Fetcher"]:::fetcher
    F["Output Handler (Modes)"]:::output
    G["CLI Output: Stdio"]:::outputMode
    H["CLI Output: SSE"]:::outputMode
    I["Shared Utility Functions"]:::utility

    A --> B
    B --> C
    C -->|"fetch_url"| D
    C -->|"fetch_youtube"| E
    D --> F
    E --> F
    F --> G
    F --> H
    D -- uses --> I
    E -- uses --> I

    click A "https://github.com/egoist/fetch-mcp/blob/main/cli.js"
    click B "https://github.com/egoist/fetch-mcp/blob/main/index.ts"
    click D "https://github.com/egoist/fetch-mcp/blob/main/markdown.ts"
    click E "https://github.com/egoist/fetch-mcp/blob/main/youtube.ts"
    click I "https://github.com/egoist/fetch-mcp/blob/main/utils.ts"

    classDef cli fill:#FFDD57,stroke:#333,stroke-width:2px
    classDef core fill:#6CC,stroke:#333,stroke-width:2px
    classDef decision fill:#FFABAB,stroke:#333,stroke-width:2px,stroke-dasharray: 5,5
    classDef fetcher fill:#B4E197,stroke:#333,stroke-width:2px
    classDef output fill:#C1DFF7,stroke:#333,stroke-width:2px
    classDef outputMode fill:#ACE,stroke:#333,stroke-width:2px
    classDef utility fill:#FDE293,stroke:#333,stroke-width:2px,stroke-dasharray: 5,5
```
