# pskill9/website-downloader architecture

> Architecture diagram of the GitHub repository pskill9/website-downloader, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pskill9/website-downloader
- Repository: https://github.com/pskill9/website-downloader

## Overview

##### Project type and purpose
- This is a single‐service “MCP Server” written in TypeScript/Node.js whose sole responsibility is to download an entire website via the external `wget` binary, preserve its structure, fix links for local browsing, and output the files to a local directory.
- It is neither a multi‐tier full‐stack app nor a compiler; rather it’s a small backend microservice/tool that orchestrates an external CLI (`wget`) and exposes a simple interface to an MCP orchestrator or CLI.

##### File structure
- Top level:
- package.json / package-lock.json – Node.js project manifest
- tsconfig.json – TypeScript compiler config
- .gitignore – ignored files
- README.md – project description, usage, prerequisites
- src/index.ts – single entry point where the server logic
lives; likely parses incoming JSON, spawns `wget`, streams output or returns status.
- No “frontend”, no “database”, no multi-module layout—this is a standalone backend service.

##### README for additional insights
- Describes a CLI-like interface called `download_website` with parameters (`url`, `outputPath`, `depth`).
- Requires system‐level dependency `wget`.
- Usage within an MCP configuration: the MCP orchestrator invokes `node build/index.js` with a JSON payload.
- No mention of HTTP; interaction might be via STDIN/STDOUT or MCP’s IPC.

##### Main components
a. MCP Orchestrator
b. Website Downloader Service (Node.js process running build/index.js)
c. External CLI Dependency (`wget`)
d. Local File System (output directory)
e. Client (could be CLI or internal MCP execution engine)

##### Relationships and interactions
1. Client or MCP Orchestrator → (invoke, pass JSON) → Node.js Service
2. Node.js Service → (spawn child process) → wget CLI
3. wget CLI → (fetch & write) → Local File System
4. Node.js Service ← (exit code / stdout/stderr) ← wget CLI
5. Node.js Service → (return status, logs) → Client / MCP Orchestrator

##### Architectural patterns and design principles
- Single‐responsibility microservice: one process → one job.
- Process orchestration: Node.js manages an external process.
- “Adapter” pattern: Node.js adapts MCP’s JSON API to wget’s command‐line interface.
- No persistent database—uses local file system as “storage”.

##### Key technologies and libraries
- Node.js & TypeScript
- child_process (Node.js) to spawn `wget`
- system’s `wget` binary (external)
- (Optional) any logging framework in index.ts

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client and Orchestrator
    Client["Client / CLI"]:::client
    Orch["MCP Orchestrator"]:::compute

    %% Service and External Tools
    Service["Node.js Website-Downloader Service"]:::compute
    Wget["wget CLI"]:::external
    FS["Local File System"]:::storage

    %% Source Code Files
    subgraph "Codebase"
        FileIndex["src/index.ts"]:::code
        PackageJSON["package.json"]:::code
        TSConfig["tsconfig.json"]:::code
        Readme["README.md"]:::code
    end

    %% Data and Control Flows
    Client -->|"invokes with JSON payload"| Orch
    Orch -->|"passes JSON payload"| Service
    Service -->|"spawn child process with CLI args"| Wget
    Wget -->|"writes site files"| FS
    Wget -->|"exit code & stdout/stderr"| Service
    Service -->|"returns status/logs"| Orch

    %% Click Events
    click FileIndex "https://github.com/pskill9/website-downloader/blob/main/src/index.ts"
    click PackageJSON "https://github.com/pskill9/website-downloader/blob/main/package.json"
    click TSConfig "https://github.com/pskill9/website-downloader/blob/main/tsconfig.json"
    click Readme "https://github.com/pskill9/website-downloader/blob/main/README.md"

    %% Styles
    classDef compute fill:#cce5ff,stroke:#339,stroke-width:1px
    classDef external fill:#ffe5cc,stroke:#e65,stroke-width:1px
    classDef storage fill:#ccffcc,stroke:#393,stroke-width:1px
    classDef client fill:#f2f2f2,stroke:#666,stroke-width:1px
    classDef code fill:#f9f9f9,stroke:#999,stroke-width:1px,font-style:italic
```
