# xixu-me/xget architecture

> Architecture diagram of the GitHub repository xixu-me/xget, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xixu-me/xget
- Repository: https://github.com/xixu-me/xget

## Overview

##### Project Type and Purpose
- Xget is a backend-only, high-performance HTTP proxy/accelerator service implemented in Node.js.
- It runs as a serverless API (Cloudflare Workers or Vercel serverless functions) or as a containerized Node.js app (Docker).
- Its core job is to accept incoming download/Git/API requests, transform URLs according to a platform config (GitHub, npm, PyPI, container registries, AI providers, etc.), fetch the resource from the upstream host with retries, caching and compression, then stream it back to the client with security headers and performance metrics.

##### File Structure Highlights
- Root: server.js (entrypoint for Node), Dockerfile, wrangler.toml (Cloudflare), vercel.json (Vercel).
- api/[…path].js & api/health.js: serverless handlers routing all incoming requests.
- src/index.js, src/config/{index.js,platforms.js}: main logic – environment/config loading, platform map, request transformation.
- PerformanceMonitor, retry & timeout logic live under src (imported by api handlers).
- Tests in test/, scripts in scripts/, CI definitions under .github/workflows.

##### Key Architectural Patterns
- API Gateway / Router: Single HTTP entry, dynamic path mapping via catch-all route.
- Strategy / Plugin Pattern: Platform definitions in src/config/platforms.js, each with its own base URL and transform function.
- Proxy + Streaming: Upstream fetch with HTTP/3, gzip/deflate/brotli, range requests, then pipe response.
- Caching Layer: Cloudflare Cache API at the edge, 30-minute TTL by default, with bypass rules for Git operations.
- Observability: PerformanceMonitor class instruments each stage (DNS, connect, first byte, end) and returns X-Performance-Metrics.
- Security Middleware: Request validation (method whitelist, path length limit), injection of security headers (CSP, HSTS, X-Frame, X-XSS, Referrer-Policy).

A. Clients
- Git clients (git clone/pull/push)
- Download tools (wget, curl, aria2, IDM)
- Browsers (via extensions or direct HTTP)
- CI/CD runners (GitHub Actions, GitLab CI)
B. CDN / Edge Cache (Cloudflare Workers + Cache API)
- Performs routing at nearest PoP, applies caching and security headers.
C. Serverless/API Layer
- api/[…path].js: universal catch-all for HTTP requests
- api/health.js: health-check endpoint
- server.js: Express-style wrapper for self-hosted mode
D. Core Service Modules
1. Router/Controller
- Parses the incoming path, extracts platform prefix
2. Config & Platform Strategy
- src/config/platforms.js maps prefixes → base URLs + transform logic
3. Fetcher / Proxy Engine
- Handles HTTP/3, multi-range, parallel download, retry logic
4. Cache Manager
- Integrates with Cloudflare Cache API for reading/writing edge cache
5. Performance Monitor
- Instruments timing and emits X-Performance-Metrics
6. Security Layer
- Validates methods/paths, injects security headers
E. External Upstream Services
- Git endpoints (GitHub, GitLab, Gitea, AOSP)
- Package registries (npm, PyPI, Maven, etc.)
- Container registries (docker, ghcr, gcr, etc.)
- AI inference APIs (OpenAI, Anthropic, HuggingFace, etc.)
- Other downloads (Apache, CTAN, arXiv, F-Droid, Linux distros)

##### Relationships & Data Flow
1. Client → DNS → CDN (Cloudflare PoP)
2. CDN Worker (api/[…path].js) → Security Layer → Router
3. Router determines platform config → Fetcher
4. Fetcher calls Upstream Service (with retry, timeout)
5. Response streamed back to Cache Manager → Edge Cache & Performance Monitor
6. CDN returns response to Client with security headers and X-Performance-Metrics

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients
    subgraph "Clients" 
        style Clients fill:#DDDDDD,stroke:#888,stroke-width:1px
        GC["Git Clients"]:::client
        DT["Download Tools"]:::client
        BR["Browsers"]:::client
        CI["CI/CD Runners"]:::client
    end

    %% Edge / CDN Layer
    CDN["CDN / Edge Cache"]:::edge

    %% Deployment Options
    subgraph "Hosting / Deployment" 
        style Hosting fill:#F9F9F9,stroke:#AAA,stroke-width:1px
        CF["Cloudflare Workers"]:::edge
        VZ["Vercel Serverless"]:::edge
        DC["Docker Container"]:::edge
    end

    %% Serverless/API Layer
    subgraph "Serverless / API Layer" 
        style Serverless fill:#E0F7FA,stroke:#0288D1,stroke-width:2px
        subgraph "Handlers"
            CH["Catch-all Handler"]:::internal
            HC["Health Check Endpoint"]:::internal
        end
        SH["Self-hosted Entrypoint"]:::internal
    end

    %% Core Service Modules
    subgraph "Core Service Modules" 
        style Core fill:#BBDEFB,stroke:#1976D2,stroke-width:2px
        RM["Router / Controller"]:::internal
        PS["Platform Strategy"]:::internal
        FE["Fetcher / Proxy Engine"]:::internal
        CM["Cache Manager"]:::internal
        PM["Performance Monitor"]:::internal
        SM["Security Middleware"]:::internal
    end

    %% Upstream Services
    subgraph "Upstream Services" 
        style Upstream fill:#E8F5E9,stroke:#388E3C,stroke-width:2px
        UE1["Git Endpoints"]:::external
        UE2["Package Registries"]:::external
        UE3["Container Registries"]:::external
        UE4["AI Providers"]:::external
        UE5["Other Downloads"]:::external
    end

    %% Connections - Request Flow
    GC -->|requests| CDN
    DT -->|requests| CDN
    BR -->|requests| CDN
    CI -->|requests| CDN

    CDN --> CF
    CDN --> VZ
    CDN --> DC

    CF -->|invokes| CH
    VZ -->|invokes| CH
    DC -->|invokes| SH

    CH -->|pass to| SM
    HC -->|health check| SH

    SM -->|validated request| RM
    RM -->|routes| PS
    PS -->|config lookup| PS
    PS -->|transforms URL| FE

    FE -->|fetches| UE1
    FE -->|fetches| UE2
    FE -->|fetches| UE3
    FE -->|fetches| UE4
    FE -->|fetches| UE5

    UE1 -->|streams| FE
    UE2 -->|streams| FE
    UE3 -->|streams| FE
    UE4 -->|streams| FE
    UE5 -->|streams| FE

    FE -->|stream| CM
    FE -->|instrument| PM

    CM -.->|cache write| CDN
    CM -.->|cache read| CDN

    PM -->|adds metrics| CDN
    SM -->|inject headers| CDN

    CDN -->|response| GC
    CDN -->|response| DT
    CDN -->|response| BR
    CDN -->|response| CI

    %% Click Events
    click CH "https://github.com/xixu-me/xget/blob/main/api/[...path].js"
    click HC "https://github.com/xixu-me/xget/blob/main/api/health.js"
    click SH "https://github.com/xixu-me/xget/blob/main/server.js"
    click PS "https://github.com/xixu-me/xget/blob/main/src/config/platforms.js"
    click RM "https://github.com/xixu-me/xget/blob/main/src/config/index.js"
    click FE "https://github.com/xixu-me/xget/blob/main/src/index.js"
    click PM "https://github.com/xixu-me/xget/blob/main/src/index.js"
    click SM "https://github.com/xixu-me/xget/blob/main/src/index.js"
    click wrangler "https://github.com/xixu-me/xget/blob/main/wrangler.toml"
    click VZ "https://github.com/xixu-me/xget/blob/main/vercel.json"
    click DC "https://github.com/xixu-me/xget/tree/main/Dockerfile"
    click CI "https://github.com/xixu-me/xget/tree/main/Client CI/CD Runners" "N/A"
    click .github "https://github.com/xixu-me/xget/blob/main/ .github/workflows/image.yml"
    click Test "https://github.com/xixu-me/xget/tree/main/test/"
    click Bench "https://github.com/xixu-me/xget/blob/main/test/benchmark/performance.bench.js"
    click Utils1 "https://github.com/xixu-me/xget/blob/main/test/helpers/test-utils.js"
    click Utils2 "https://github.com/xixu-me/xget/blob/main/test/utils.test.js"
    click PackageJSON "https://github.com/xixu-me/xget/blob/main/package.json"
    click PackageLock "https://github.com/xixu-me/xget/blob/main/package-lock.json"

    %% Styles
    classDef client fill:#EEEEEE,stroke:#AAA,stroke-width:1px;
    classDef edge fill:#FFE0B2,stroke:#FB8C00,stroke-width:1px;
    classDef internal fill:#BBDEFB,stroke:#1976D2,stroke-width:1px;
    classDef external fill:#C8E6C9,stroke:#388E3C,stroke-width:1px;
```
