# swiftpackageindex/swiftpackageindex-server architecture

> Architecture diagram of the GitHub repository swiftpackageindex/swiftpackageindex-server, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/swiftpackageindex/swiftpackageindex-server
- Repository: https://github.com/swiftpackageindex/swiftpackageindex-server

## Overview

##### Project Type & Purpose
- Full-stack web application: A site for discovering and evaluating Swift Package Manager packages.
- Backend written in Swift using Vapor (MVC pattern, controllers, models, services).
- Frontend: static HTML/CSS/JS assets (built with esbuild and Sass) enhancing site interactivity.
- Integrates with external services (GitHub, GitLab, Plausible analytics), uses object storage (S3), a relational database (Postgres), and a cache (Redis).

##### File Structure Overview
- FrontEnd/: source assets (JavaScript controllers, SCSS styles, build script).
- Public/: compiled assets (images, robots.txt).
- Sources/App/: Vapor application (Commands, Controllers/API & Site, Core services, Models, Views, Migrations).
- S3Store/: wrapper for S3 interactions.
- Scripts/: helper scripts for ingestion loops, CI DB setup, emoji updates.
- infrastructure/: Prometheus/Loki/Promtail configs for monitoring/log aggregation.
- CI/CD configs: GitHub Actions (.github/workflows), GitLab CI, Dockerfile, devcontainer.

a. Frontend Layer
– Browser clients request pages and API endpoints.
– Static assets served via a CDN or directly by the Vapor app.
– JavaScript controllers manage dynamic UI (search, charts, modals).
b. API/Web Layer
– Vapor HTTP server (entrypoint → routes → controllers).
– Controllers orchestrate requests:
- API controllers (JSON endpoints for packages, search, collections).
- Site controllers (render HTML via Leaf-like Views).
c. Core Services Layer
– Authentication, Metrics, Logging, Error Handling Middleware.
– Dependency‐injected clients: HTTPClient (for GitHub/GitLab), RedisClient, S3Client, ShellClient.
– Background commands (ingestion, build triggers, reconciliation) executed via CLI commands or scheduled loops.
d. Data Layer
– PostgreSQL database (persistent storage for packages, builds, versions, migrations).
– Redis (caching, rate‐limiting, reference lookups).
– S3 Object Storage (readme HTML, doc archives).
e. External Integrations
– GitHub/GitLab APIs for repository metadata & build logs.
– Plausible analytics for event tracking.
– Package Collections sign/verify process.
f. Infrastructure & Observability
– Containerization (Docker, devcontainer).
– CI/CD pipelines (GitHub Actions, GitLab CI).
– Monitoring: Prometheus (metrics), Loki/Promtail (logs).

##### Architectural Patterns & Design Principles
- MVC: Controllers (entrypoints) → Models (data) → Views (HTML templates).
- Dependency Injection for pluggable clients (e.g., RedisClient).
- Command Pattern: CLI Commands encapsulate background tasks.
- Layered Architecture: clear separation between web/API, business logic, persistence.
- Eventual consistency: background ingestion and reconciliation ensure up‐to‐date data.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend Layer
    subgraph "Frontend Layer"
        direction TB
        Browser["Browser Client"]:::frontend
        FrontEndBuild["Source Assets & Build Pipeline"]:::frontend
        PublicAssets["Compiled Static Assets"]:::frontend
        JSControllers["JavaScript Controllers"]:::frontend
    end

    %% Vapor App Layer
    subgraph "Vapor App"
        direction TB
        RunEntrypoint["Run Entrypoint"]:::app
        Configure["Configuration Setup"]:::app
        Routes["Routes Definition"]:::app
        RoutesDoc["Routes Documentation"]:::app
        subgraph "Controllers (MVC)"
            direction TB
            APIControllers["API Controllers"]:::app
            SiteControllers["Site Controllers"]:::app
        end
    end

    %% Core Services Layer
    subgraph "Core Services & Middleware"
        direction TB
        CoreServices["Core Services"]:::app
        Dependencies["Injected Clients"]:::app
        ErrorMiddleware["Error Handling Middleware"]:::app
        BackendReporting["Backend Reporting Middleware"]:::app
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        Models["Persistent Models"]:::data
        Migrations["Database Migrations"]:::data
        Postgres["PostgreSQL Database"]:::data
        RedisCache["Redis Cache"]:::data
        S3ObjectStorage["S3 Object Storage"]:::data
    end

    %% Background Workers
    subgraph "Background Workers"
        direction TB
        CLICommands["CLI Commands"]:::app
        IngestLoop["Ingestion Loop Script"]:::app
    end

    %% External Services
    subgraph "External Services"
        direction TB
        GitHubAPI["GitHub API"]:::external
        GitLabAPI["GitLab API"]:::external
        Plausible["Plausible Analytics"]:::external
        PackageCollections["Package Collections"]:::external
    end

    %% Infrastructure & CI/CD
    subgraph "Infrastructure & CI/CD"
        direction TB
        Prometheus["Prometheus Metrics"]:::infra
        LokiPromtail["Loki/Promtail Logs"]:::infra
        GitHubActions["GitHub Actions"]:::infra
        GitLabCI["GitLab CI"]:::infra
        DockerEnv["Docker & Devcontainer"]:::infra
    end

    %% Connections
    Browser -->|HTTP Request| RunEntrypoint
    Browser --> FrontEndBuild
    Browser --> PublicAssets
    Browser --> JSControllers
    JSControllers -->|fetch /api| APIControllers

    RunEntrypoint --> Configure
    Configure --> Routes
    Routes --> APIControllers
    Routes --> SiteControllers
    RoutesDoc --> Routes

    APIControllers --> CoreServices
    SiteControllers --> CoreServices

    CoreServices --> Dependencies
    CoreServices --> ErrorMiddleware
    CoreServices --> BackendReporting

    CoreServices --> Postgres
    CoreServices --> RedisCache
    CoreServices --> S3ObjectStorage

    CLICommands --> GitHubAPI
    CLICommands --> GitLabAPI
    IngestLoop --> CLICommands

    CoreServices --> Prometheus
    CoreServices --> LokiPromtail

    GitHubActions --> DockerEnv
    GitLabCI --> DockerEnv

    %% Click Events
    click FrontEndBuild "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/FrontEnd/"
    click PublicAssets "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/Public/"
    click RunEntrypoint "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/Run/entrypoint.swift"
    click Configure "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/Sources/App/configure.swift"
    click Routes "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/Sources/App/routes.swift"
    click RoutesDoc "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/Sources/App/routes+documentation.swift"
    click APIControllers "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/Sources/App/Controllers/API/"
    click SiteControllers "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/Sources/App/Controllers/"
    click CoreServices "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/Sources/App/Core/"
    click Dependencies "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/Sources/App/Core/Dependencies/"
    click ErrorMiddleware "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/Sources/App/Core/ErrorMiddleware.swift"
    click BackendReporting "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/Sources/App/Core/BackendReportingMiddleware.swift"
    click Models "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/Sources/App/Models/"
    click Migrations "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/Sources/App/Migrations/"
    click S3ObjectStorage "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/S3Store/S3Store.swift"
    click CLICommands "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/Sources/App/Commands/"
    click IngestLoop "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/scripts/ingest-loop.sh"
    click LokiPromtail "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/infrastructure/"
    click GitHubActions "https://github.com/swiftpackageindex/swiftpackageindex-server/tree/main/.github/workflows/"
    click GitLabCI "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/.gitlab-ci.yml"
    click DockerEnv "https://github.com/swiftpackageindex/swiftpackageindex-server/blob/main/.devcontainer/devcontainer.json"

    %% Styles
    classDef frontend fill:#add8e6,stroke:#333,stroke-width:1px
    classDef app fill:#90ee90,stroke:#333,stroke-width:1px
    classDef data fill:#ffa500,stroke:#333,stroke-width:1px
    classDef external fill:#dda0dd,stroke:#333,stroke-width:1px
    classDef infra fill:#d3d3d3,stroke:#333,stroke-width:1px
```
