# ray-d-song/web-archive architecture

> Architecture diagram of the GitHub repository ray-d-song/web-archive, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ray-d-song/web-archive
- Repository: https://github.com/ray-d-song/web-archive

## Overview

- Browser Plugin – for saving web pages as single HTML files and uploading them.
- Server – built on Cloudflare Workers that accepts the uploads, processes them, stores them in a Cloudflare D1 database and R2 storage bucket, and exposes APIs.
- Web Client – for users to search, view, and share the archived pages.

##### Main Components
a. Browser Plugin:
– Located under “packages/plugin”. It includes directories for background scripts, content scripts, popup UI (with React/TSX components), internationalization, and various helper libraries.
b. Server:
– Found in “packages/server”. It contains API endpoints (auth, folders, pages, data, etc.), middleware for token validation, database models, SQL migrations, and utility functions. This server runs on Cloudflare Workers, leveraging Cloudflare D1 (SQL database) and R2 (object storage bucket).
c. Web Client:
– Located in “packages/web”. It has the web entry point, pages, components, routing, state management (store), asset management, and configuration files (postcss, tailwind, vite).

##### Out Relationships and Data Flow
a. The Browser Plugin is responsible for capturing a webpage, converting it into a single HTML file, and then uploading it via API calls.
b. The Server receives this HTML file. It then:
– Processes authentication (via a token middleware).
– Stores meta-information and metadata into the D1 database.
– Saves the full HTML file in the R2 storage bucket.
c. The Web Client communicates with the Server API to query archive records, display content, and search or filter through folders, tags, and other metadata.
d. There are shared components and utilities under “packages/shared” indicating that some UI components or helper functions are reused between the Web Client and Plugin (“DRY” principle).

##### Architectural Patterns and Design Principles
a. Separation of Concerns: Each part – Plugin, Server, and Client – has its own dedicated directory and clear responsibilities.
b. Modular Design: The project is split into packages (monorepo structure using pnpm-workspace).
c. Reusability: The “packages/shared” folder contains common UI components and utilities that are shared across different parts.
d. Cloud-based Serverless Architecture: The Server part leverages Cloudflare’s serverless functions, along with managed services (D1 and R2).

– Browser Plugin (Client-Side Extension)
– Server (Cloudflare Worker with API endpoints, authentication, processing logic, and interaction with storage)
– Web Client (User Interface Web Application)
– Underlying Systems: Cloudflare D1 database and Cloudflare R2 storage bucket.
– Plugin: Highlight Background, Content Scripts, Popup UI, and the internationalization support.
– Web Client: Outline the routing, state management (store modules), and UI components.
– From Browser Plugin → Server: Upload API calls (HTML file and metadata).
– From Web Client → Server: Queries and API calls to retrieve archived pages.

##### Final Elements
- The Plugin sending “save webpage” requests to the Server.
- The Server writing metadata to D1 and storing HTML files in R2.
- The Web Client fetching and displaying data from the Server.
– Include annotations (or legends) explaining architectural patterns, technology choices (e.g., Cloudflare Workers, pnpm workspace, React, Tailwind, etc.), and any interdependencies.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Browser Plugin"
        BP1("Background Scripts"):::plugin
        BP2("Content Scripts"):::plugin
        BP3("Popup UI"):::plugin
        BP4("Internationalization"):::plugin
        BP5("Helper Libraries"):::plugin
        BP6("Helper Utilities"):::plugin
    end

    subgraph "Server (Cloudflare Workers)"
        S1("API Endpoints"):::server
        S2("Middleware"):::server
        S3("Database Models"):::server
        S4("SQL Migrations"):::server
        S5("Utility Functions"):::server
        S6("Wrangler Config"):::server
        S7("Wrangler Raw Config"):::server
    end

    subgraph "Web Client"
        WC1("Web Entry"):::client
        WC2("Pages & Routing"):::client
        WC3("UI Components"):::client
        WC4("State Management"):::client
        WC5("PostCSS Config"):::client
        WC6("Tailwind Config"):::client
        WC7("Vite Config"):::client
    end

    subgraph "Shared Modules"
        SM1("Common UI Components"):::shared
        SM2("Shared Utilities"):::shared
        SM3("Shared Types"):::shared
    end

    subgraph "Infrastructure"
        DB("Cloudflare D1 Database"):::database
        R2("Cloudflare R2 Storage Bucket"):::storage
    end

    %% Data Flow Connections
    BP3 -->|"UploadHTML"| S1
    WC1 -->|"QueryArchive"| S1
    S1 -->|"StoreMetadata"| DB
    S1 -->|"StoreHTML"| R2

    %% Shared Modules Usage
    BP3 ---|"UsesCommon"| SM1
    WC3 ---|"UsesCommon"| SM1

    %% Styles
    classDef plugin fill:#FAD4D4,stroke:#B22222,stroke-width:2px;
    classDef server fill:#D4F2D4,stroke:#228B22,stroke-width:2px;
    classDef client fill:#D4E8F2,stroke:#1E90FF,stroke-width:2px;
    classDef shared fill:#FFF2B2,stroke:#DAA520,stroke-width:2px;
    classDef database fill:#FFE4B5,stroke:#FF8C00,stroke-width:2px;
    classDef storage fill:#E6E6FA,stroke:#6A5ACD,stroke-width:2px;

    %% Click Events - Browser Plugin
    click BP1 "https://github.com/ray-d-song/web-archive/tree/main/packages/plugin/background"
    click BP2 "https://github.com/ray-d-song/web-archive/tree/main/packages/plugin/contentScripts"
    click BP3 "https://github.com/ray-d-song/web-archive/tree/main/packages/plugin/popup"
    click BP4 "https://github.com/ray-d-song/web-archive/tree/main/packages/plugin/assets/locales"
    click BP5 "https://github.com/ray-d-song/web-archive/tree/main/packages/plugin/lib"
    click BP6 "https://github.com/ray-d-song/web-archive/tree/main/packages/plugin/utils"

    %% Click Events - Server
    click S1 "https://github.com/ray-d-song/web-archive/tree/main/packages/server/src/api"
    click S2 "https://github.com/ray-d-song/web-archive/tree/main/packages/server/src/middleware"
    click S3 "https://github.com/ray-d-song/web-archive/tree/main/packages/server/src/model"
    click S4 "https://github.com/ray-d-song/web-archive/tree/main/packages/server/src/sql/migrations"
    click S5 "https://github.com/ray-d-song/web-archive/tree/main/packages/server/src/utils"
    click S6 "https://github.com/ray-d-song/web-archive/blob/main/packages/server/wrangler.toml"
    click S7 "https://github.com/ray-d-song/web-archive/blob/main/packages/server/wrangler.raw.toml"

    %% Click Events - Web Client
    click WC1 "https://github.com/ray-d-song/web-archive/blob/main/packages/web/index.html"
    click WC2 "https://github.com/ray-d-song/web-archive/blob/main/packages/web/src/router.ts"
    click WC3 "https://github.com/ray-d-song/web-archive/tree/main/packages/web/src/components"
    click WC4 "https://github.com/ray-d-song/web-archive/tree/main/packages/web/src/store"
    click WC5 "https://github.com/ray-d-song/web-archive/blob/main/packages/web/postcss.config.js"
    click WC6 "https://github.com/ray-d-song/web-archive/blob/main/packages/web/tailwind.config.js"
    click WC7 "https://github.com/ray-d-song/web-archive/blob/main/packages/web/vite.config.ts"

    %% Click Events - Shared Modules
    click SM1 "https://github.com/ray-d-song/web-archive/tree/main/packages/shared/components"
    click SM2 "https://github.com/ray-d-song/web-archive/tree/main/packages/shared/utils"
    click SM3 "https://github.com/ray-d-song/web-archive/tree/main/packages/shared/types"
```
