# edgelesspe/nlm architecture

> Architecture diagram of the GitHub repository edgelesspe/nlm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/edgelesspe/nlm
- Repository: https://github.com/edgelesspe/nlm

## Overview

##### Main Components
a. Frontend (Vue 3 + TypeScript SPA)
b. Backend Web Server (Go REST API)
c. Database (PostgreSQL with JSONB fields)
d. External Storage & Download Services (rclone uploader, nginx downloader)
e. External Integrations (GitHub API, Webhooks, Bot messaging service)
f. Scheduler / Trigger subsystem (Cron jobs, incoming webhooks)

##### Logical Layers in the Backend
- API Layer (HTTP handlers in `backend/handler`)
- Middleware Layer (`backend/middleware`)
- Service Layer (`backend/service`)
- Pipeline Orchestration (`backend/pipeline`)
- Domain & Model (`backend/domain`, `backend/model`)
- Persistence Layer (`backend/db`)
- Driver Layer (`backend/driver`) for external I/O
- Utility Layer (`backend/utils`) for common helpers
- VO Layer (`backend/vo`) for transport/value objects

##### Interactions and Data Flows
a. Frontend → HTTP(S) requests → Backend API
b. Middleware authenticates/validates → Handlers dispatch to Services
c. Service logic may:
- Read/write from Database
- Invoke Drivers (download via nginx, upload via rclone)
- Trigger or advance Pipelines (ept, bot, QA, mirror, storage)
- Call external APIs (GitHub via `backend/utils/github.go`)
- Emit webhooks or bot notifications (`service/bot.go`, `handler/webhook.go`)
d. Scheduler (Cron) triggers scheduled pipelines (`trigger/cron.go`)
e. Incoming Webhooks hit `trigger/webhook.go` → injected into pipeline

##### External Components & Data Stores
- PostgreSQL: tables for NEP, EPT, pipelines, storage records
- Nginx Server: serves download URLs for mirrored packages
- Rclone‐backed Storage: pushes artifacts to S3/GCS/etc.
- GitHub API: fetch release metadata, validate packages
- Messaging Bot: Telegram/Slack notifications for pipeline results

##### Architectural Patterns & Principles
- Layered Architecture: clear separation between API, service, domain, persistence
- Dependency Injection: config/env loaded in `backend/config/env.go`
- Modular Pipelines: each pipeline type in `backend/pipeline`
- Trigger‐Driven Workflows: cron + webhook subsystems feed into pipelines
- VO & Model segregation: transport vs. DB schema

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client Layer
    subgraph "Client / Browser"
        direction TB
        FEApp["Vue SPA"]:::frontend
        FEAPI["API Client"]:::frontend
        Views["Views & Components"]:::frontend
    end

    %% Backend Service
    subgraph "Backend Service"
        direction TB
        Main["main.go"]:::core
        subgraph "Config & Env"
            EnvLoader["Env Loader"]:::infra
            EnvExample[".env.example.toml"]:::infra
        end
        subgraph "API Layer"
            direction TB
            HandlerNEP["Handler: NEP"]:::core
            HandlerStorage["Handler: Storage"]:::core
            HandlerPipeline["Handler: Pipeline"]:::core
            HandlerMirror["Handler: Mirror"]:::core
            HandlerWebhook["Handler: Webhook"]:::core
        end
        subgraph "Middleware Layer"
            Middleware["Middleware"]:::core
        end
        subgraph "Service Layer"
            direction TB
            ServiceNEP["Service: NEP"]:::core
            ServiceStorage["Service: Storage"]:::core
            ServiceQA["Service: QA"]:::core
            ServiceMirrorEPT["Service: Mirror/EPT"]:::core
            ServiceBot["Service: Bot"]:::core
        end
        subgraph "Pipeline Orchestration"
            PipelineEPT["Pipeline: EPT"]:::core
            PipelineBot["Pipeline: Bot"]:::core
            PipelineTypes["Pipeline Types"]:::core
        end
        subgraph "Trigger Subsystem"
            TriggerCron["Trigger: Cron"]:::core
            TriggerWebhook["Trigger: Webhook"]:::core
        end
        subgraph "Domain & Model"
            DomainNEP["Domain: NEP"]:::core
            ModelNEP["Model: NEP"]:::core
            ModelEPT["Model: EPT"]:::core
            ModelPipeline["Model: Pipeline"]:::core
            ModelStorage["Model: Storage"]:::core
        end
        subgraph "Persistence Layer"
            DBConn["DB Connection"]:::infra
            JSONB["JSONB Helpers"]:::infra
        end
        subgraph "Driver Layer"
            Downloader["Driver: Nginx Downloader"]:::integration
            Uploader["Driver: Rclone Uploader"]:::integration
        end
        subgraph "Utility Layer"
            GitHubUtil["GitHub Client"]:::integration
            FileUtil["File Helpers"]:::integration
            HashUtil["Hash/HMAC"]:::integration
            CompressUtil["Compression"]:::integration
        end
        subgraph "VO Layer"
            VO["Value Objects"]:::core
        end
    end

    %% External Systems
    subgraph "External Services"
        direction TB
        Postgres["PostgreSQL"]:::infra
        Nginx["Nginx Server"]:::infra
        RcloneStorage["Rclone Storage"]:::infra
        GitHubAPI["GitHub API"]:::infra
        BotMsg["Messaging Bot"]:::infra
    end

    %% Flows
    User(("User")):::frontend -->|Interact| FEApp
    FEApp -->|HTTP| FEAPI
    FEAPI -->|HTTP| Main
    Main --> HandlerNEP
    Main --> HandlerStorage
    Main --> HandlerPipeline
    Main --> HandlerMirror
    Main --> HandlerWebhook

    HandlerNEP --> Middleware --> ServiceNEP
    HandlerStorage --> Middleware --> ServiceStorage
    HandlerPipeline --> Middleware --> ServiceQA
    HandlerMirror --> Middleware --> ServiceMirrorEPT
    HandlerWebhook --> Middleware --> ServiceBot

    ServiceNEP --> DBConn
    ServiceStorage --> DBConn
    ServiceQA --> PipelineEPT
    ServiceMirrorEPT --> PipelineEPT
    ServiceMirrorEPT --> Downloader
    ServiceMirrorEPT --> Uploader
    ServiceBot --> BotMsg

    PipelineEPT -->|orchestrate| JSONB
    PipelineBot --> ServiceBot

    TriggerCron --> PipelineEPT
    TriggerWebhook --> PipelineBot

    DBConn --> Postgres
    JSONB --> Postgres

    GitHubUtil --> GitHubAPI
    ServiceMirrorEPT --> GitHubUtil

    Nginx --> Downloader
    RcloneStorage --> Uploader

    %% Click Events
    click FEApp "https://github.com/edgelesspe/nlm/blob/master/frontend/src/main.ts"
    click FEAPI "https://github.com/edgelesspe/nlm/blob/master/frontend/src/api/index.ts"
    click nep.ts "https://github.com/edgelesspe/nlm/blob/master/frontend/src/api/nep.ts"
    click storage.ts "https://github.com/edgelesspe/nlm/blob/master/frontend/src/api/storage.ts"
    click Views "https://github.com/edgelesspe/nlm/tree/master/frontend/src/views/"
    click Views "https://github.com/edgelesspe/nlm/tree/master/frontend/src/components/"
    click Main "https://github.com/edgelesspe/nlm/blob/master/backend/main.go"
    click EnvLoader "https://github.com/edgelesspe/nlm/blob/master/backend/config/env.go"
    click EnvExample "https://github.com/edgelesspe/nlm/blob/master/backend/.env.example.toml"
    click HandlerNEP "https://github.com/edgelesspe/nlm/blob/master/backend/handler/nep.go"
    click HandlerStorage "https://github.com/edgelesspe/nlm/blob/master/backend/handler/storage.go"
    click HandlerPipeline "https://github.com/edgelesspe/nlm/blob/master/backend/handler/pipeline.go"
    click HandlerMirror "https://github.com/edgelesspe/nlm/blob/master/backend/handler/mirror.go"
    click HandlerWebhook "https://github.com/edgelesspe/nlm/blob/master/backend/handler/webhook.go"
    click Middleware "https://github.com/edgelesspe/nlm/blob/master/backend/middleware/index.go"
    click ServiceNEP "https://github.com/edgelesspe/nlm/blob/master/backend/service/nep.go"
    click ServiceStorage "https://github.com/edgelesspe/nlm/blob/master/backend/service/storage.go"
    click ServiceQA "https://github.com/edgelesspe/nlm/blob/master/backend/service/qa.go"
    click ServiceMirrorEPT "https://github.com/edgelesspe/nlm/blob/master/backend/service/mirror-ept.go"
    click ServiceMirrorEPT "https://github.com/edgelesspe/nlm/blob/master/backend/service/mirror-pkg.go"
    click ServiceMirrorEPT "https://github.com/edgelesspe/nlm/blob/master/backend/service/mirror.go"
    click ServiceBot "https://github.com/edgelesspe/nlm/blob/master/backend/service/bot.go"
    click PipelineEPT "https://github.com/edgelesspe/nlm/blob/master/backend/pipeline/ept.go"
    click PipelineBot "https://github.com/edgelesspe/nlm/blob/master/backend/pipeline/bot.go"
    click PipelineTypes "https://github.com/edgelesspe/nlm/blob/master/backend/pipeline/type.go"
    click TriggerCron "https://github.com/edgelesspe/nlm/blob/master/backend/trigger/cron.go"
    click TriggerWebhook "https://github.com/edgelesspe/nlm/blob/master/backend/trigger/webhook.go"
    click DomainNEP "https://github.com/edgelesspe/nlm/blob/master/backend/domain/nep.go"
    click ModelNEP "https://github.com/edgelesspe/nlm/blob/master/backend/model/nep.go"
    click ModelEPT "https://github.com/edgelesspe/nlm/blob/master/backend/model/ept.go"
    click ModelPipeline "https://github.com/edgelesspe/nlm/blob/master/backend/model/pipeline.go"
    click ModelStorage "https://github.com/edgelesspe/nlm/blob/master/backend/model/storage.go"
    click DBConn "https://github.com/edgelesspe/nlm/blob/master/backend/db/index.go"
    click JSONB "https://github.com/edgelesspe/nlm/blob/master/backend/db/jsonb.go"
    click Downloader "https://github.com/edgelesspe/nlm/blob/master/backend/driver/downloader/nginx.go"
    click Uploader "https://github.com/edgelesspe/nlm/blob/master/backend/driver/uploader/rclone.go"
    click GitHubUtil "https://github.com/edgelesspe/nlm/blob/master/backend/utils/github.go"
    click FileUtil "https://github.com/edgelesspe/nlm/blob/master/backend/utils/fs.go"
    click FileUtil "https://github.com/edgelesspe/nlm/blob/master/backend/utils/download.go"
    click HashUtil "https://github.com/edgelesspe/nlm/blob/master/backend/utils/blake3.go"
    click HashUtil "https://github.com/edgelesspe/nlm/blob/master/backend/utils/hmac.go"
    click CompressUtil "https://github.com/edgelesspe/nlm/blob/master/backend/utils/zstd.go"
    click VO "https://github.com/edgelesspe/nlm/blob/master/backend/vo/index.go"

    %% Styles
    classDef frontend fill:#BBDEFB,stroke:#1E88E5
    classDef core fill:#C8E6C9,stroke:#388E3C
    classDef infra fill:#FFF9C4,stroke:#FBC02D
    classDef integration fill:#FFE0B2,stroke:#F57C00
```
