# spotky1004/auto-web-archiver architecture

> Architecture diagram of the GitHub repository spotky1004/auto-web-archiver, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/spotky1004/auto-web-archiver
- Repository: https://github.com/spotky1004/auto-web-archiver

## Overview

##### Main Components

A. Browser-side Components:
- Chrome Extension (Frontend)
- Manifest.json (Extension configuration)
- Service Worker (Background processes)
- Index.html (Archive viewer)

B. Server-side Components:
- Archive Server (Node.js)
- Queue System (ArchiveQueue.js)
- Storage Management
- Page Archiver (archivePage.js)

C. Storage:
- Local File System Storage
- Archive Directory Structure

**┌─ Chrome Browser ─────────────────┐**
- ┌─ Extension UI ─┐ ┌─ Webpage ─┐│
- Archive       │ │           ││
- Viewer        │ │           ││
- ┘ └───────────┘│
- ┘

**┌─ Service Worker ─────┐  ┌─ Archive Server ────┐**
- Page Monitoring   │→→│  - Queue Management │
- Request Handling  │  │  - Page Processing  │
- ┘  └──────────────────┘

**┌─ File System ──────────────┐**
- ┌─ Archive Directory ─┐   │
- Stored Pages     │   │
- Assets          │   │
- ┘   │
- ┘

- →→ Synchronous communications
- ⇢⇢ Asynchronous operations
- ⇄ Two-way data flow

##### Important Components to Highlight

a. Archive Server Components:
- ArchiveQueue.js (Queue management)
- archivePage.js (Page processing)
- Storage management system (stroage folder)
- Utility functions (util folder)

b. Extension Components:
- Service Worker (Background processes)
- Archive Viewer (index.html)
- Extension Configuration (manifest.json)

##### Technical Details to Note

- Forever module usage for process management
- HTTP-server module for serving archived content
- Storage structure and organization
- Queue processing flow

##### The viewing capabilities for archived content

- Setup requirements
- Runtime dependencies
- System requirements
- Common troubleshooting scenarios (CORS, rendering issues)

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Chrome Browser"
        ExtUI["Extension UI"]
        WebPage["Active Webpage"]
        ArcView["Archive Viewer"]:::ui
    end

    subgraph "Extension Components"
        ManifestConfig["Extension Config"]:::config
        ServiceWorker["Service Worker"]:::worker
        click ManifestConfig "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/manifest.json"
        click ServiceWorker "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/service-worker.js"
        click ArcView "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/index.html"
    end

    subgraph "Archive Server"
        AppServer["Server Main"]:::server
        QueueSystem["Archive Queue"]:::queue
        PageArchiver["Page Archiver"]:::archiver
        click AppServer "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/app.js"
        click QueueSystem "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/src/ArchiveQueue.js"
        click PageArchiver "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/src/archivePage.js"

        subgraph "Storage Management"
            StorageSystem["Storage System"]:::storage
            CreateStorage["Create Storage"]:::storage
            GetStorage["Get Storage"]:::storage
            PathManagement["Path Management"]:::storage
            click StorageSystem "https://github.com/spotky1004/Auto-Web-Archiver/tree/main/archive-server/src/stroage/"
            click CreateStorage "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/src/stroage/createStorageItem.js"
            click GetStorage "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/src/stroage/getStorageItem.js"
            click PathManagement "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/src/stroage/getStorageItemPath.js"
        end

        subgraph "Utility Functions"
            PathCreation["Path Creation"]:::util
            DirSize["Directory Size"]:::util
            DirNameFix["Directory Name Fix"]:::util
            DirEmptyCheck["Directory Empty Check"]:::util
            DirStructure["Directory Structure"]:::util
            click PathCreation "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/util/createPath.js"
            click DirSize "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/util/dirSize.js"
            click DirNameFix "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/util/fixDirName.js"
            click DirEmptyCheck "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/util/isDirEmpty.js"
            click DirStructure "https://github.com/spotky1004/Auto-Web-Archiver/blob/main/archive-server/src/makePseudoDir.js"
        end
    end

    subgraph "File System"
        ArchiveDir["Archive Directory"]
        StoredPages["Stored Pages"]
        Assets["Assets"]
    end

    %% Relationships
    ExtUI --> ServiceWorker
    WebPage --> ServiceWorker
    ServiceWorker --> AppServer
    AppServer --> QueueSystem
    QueueSystem --> PageArchiver
    PageArchiver --> StorageSystem
    StorageSystem --> CreateStorage
    StorageSystem --> GetStorage
    StorageSystem --> PathManagement
    StorageSystem --> ArchiveDir
    ArchiveDir --> StoredPages
    ArchiveDir --> Assets
    ArcView --> GetStorage

    %% Utility Relationships
    StorageSystem --> PathCreation
    StorageSystem --> DirSize
    StorageSystem --> DirNameFix
    StorageSystem --> DirEmptyCheck
    StorageSystem --> DirStructure

    %% Styles
    classDef ui fill:#9cf
    classDef config fill:#f9f
    classDef worker fill:#fcf
    classDef server fill:#cfc
    classDef queue fill:#fcc
    classDef archiver fill:#cff
    classDef storage fill:#ffc
    classDef util fill:#ddd

    %% Legend
    subgraph Legend
        UI["UI Component"]:::ui
        Config["Configuration"]:::config
        Worker["Background Process"]:::worker
        Server["Server Component"]:::server
        Queue["Queue System"]:::queue
        Archiver["Archiver"]:::archiver
        Storage["Storage Component"]:::storage
        Util["Utility"]:::util
    end
```
