# middlewarehq/middleware architecture

> Architecture diagram of the GitHub repository middlewarehq/middleware, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/middlewarehq/middleware
- Repository: https://github.com/middlewarehq/middleware

## Overview

##### High-Level Architecture Components

The system consists of three main services:
- Web Server (Frontend - Next.js)
- Analytics Server (Backend API)
- Sync Server (Background Processing)
- Supporting Infrastructure (PostgreSQL and Redis)

##### Key Components

a) Frontend Layer:
- Next.js Web Application (port 3333)
- Client-side state management (Redux store)
- API integration layer
- Authentication system

b) Backend Services:
- Analytics Server (port 9696)
- API endpoints (/mhq/api/*)
- Service layer for metrics calculation
- Data processing modules

- Sync Server (port 9697)
- ETL processes for different providers
- Background job processing
- Data synchronization

c) Data Layer:
- PostgreSQL database
- Redis for caching/queue management

d) External Integrations:
- GitHub API
- GitLab API
- Incident Management Systems

**[Client Layer]**
- ↓ ↑
**[API Gateway/Load Balancer]**
- ↓ ↑
**[Service Layer]**
- ↓ ↑
**[Data Layer]**
- ↓ ↑
**[External Services]**

##### Important Flows to Highlight

a) Data Collection Flow:
- How the Sync Server pulls data from Git providers
- ETL pipeline for processing repository data
- Data storage flow into PostgreSQL

b) Analytics Flow:
- How DORA metrics are calculated
- Data aggregation paths
- Caching mechanisms

c) User Interaction Flow:
- Authentication process
- Team management
- Metrics visualization requests

##### Component Details

For each major component, include:
- Service name
- Port number (if applicable)
- Primary responsibilities
- Key dependencies

##### Interactions

- Data flows (thick lines)

##### Notes Section

A notes section explaining:
- Scaling considerations
- Security implementations
- Key architectural decisions
- Performance optimizations

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client
        WEB["Web Server (Next.js)<br>Port: 3333"]:::frontend
        AUTH["Authentication System"]:::frontend
        REDUX["Redux Store"]:::frontend
    end

    subgraph API_Gateway
        APIG["API Gateway"]:::api
    end

    subgraph Service_Layer
        AS["Analytics Server<br>Port: 9696"]:::backend
        SS["Sync Server<br>Port: 9697"]:::backend
        DORA["DORA Metrics<br>Analysis"]:::backend
        ETL["ETL Processing"]:::backend
    end

    subgraph Data_Layer
        DB[(PostgreSQL)]:::database
        REDIS[(Redis Cache)]:::database
        MODELS["Data Models"]:::database
    end

    subgraph External_Services
        GITHUB["GitHub API"]:::external
        GITLAB["GitLab API"]:::external
        IMS["Incident Management<br>Systems"]:::external
    end

    %% Connections
    WEB <--> |API Calls| APIG
    AUTH <--> |Auth Flows| APIG
    REDUX --> |State Updates| WEB

    APIG <--> |Routes Requests| AS
    APIG <--> |Sync Operations| SS

    AS --> |Processes| DORA
    SS --> |Manages| ETL

    AS <--> |Data Access| DB
    AS <--> |Caching| REDIS
    SS <--> |Data Storage| DB
    MODELS --- DB

    ETL <--> |Data Collection| GITHUB
    ETL <--> |Data Collection| GITLAB
    ETL <--> |Incident Data| IMS

    %% Click Events
    click WEB "https://github.com/middlewarehq/middleware/tree/main/web-server/"
    click AUTH "https://github.com/middlewarehq/middleware/tree/main/web-server/pages/api/auth/"
    click AS "https://github.com/middlewarehq/middleware/blob/main/backend/analytics_server/app.py"
    click SS "https://github.com/middlewarehq/middleware/blob/main/backend/analytics_server/sync_app.py"
    click ETL "https://github.com/middlewarehq/middleware/tree/main/backend/analytics_server/mhq/service/code/sync/"
    click DORA "https://github.com/middlewarehq/middleware/tree/main/backend/analytics_server/mhq/service/deployments/"
    click MODELS "https://github.com/middlewarehq/middleware/tree/main/backend/analytics_server/mhq/store/models/"
    click DB "https://github.com/middlewarehq/middleware/tree/main/database-docker/"
    click GITHUB "https://github.com/middlewarehq/middleware/blob/main/backend/analytics_server/mhq/exapi/github.py"
    click GITLAB "https://github.com/middlewarehq/middleware/blob/main/backend/analytics_server/mhq/exapi/gitlab.py"
    click IMS "https://github.com/middlewarehq/middleware/blob/main/backend/analytics_server/mhq/exapi/git_incidents.py"
    click APIG "https://github.com/middlewarehq/middleware/tree/main/backend/analytics_server/mhq/api/"

    %% Styles
    classDef frontend fill:#2196F3,stroke:#fff,stroke-width:2px,color:#fff
    classDef backend fill:#4CAF50,stroke:#fff,stroke-width:2px,color:#fff
    classDef database fill:#FFC107,stroke:#fff,stroke-width:2px,color:#000
    classDef external fill:#9E9E9E,stroke:#fff,stroke-width:2px,color:#fff
    classDef api fill:#FF5722,stroke:#fff,stroke-width:2px,color:#fff

    %% Legend
    subgraph Legend
        F[Frontend]:::frontend
        B[Backend]:::backend
        D[(Database)]:::database
        E[External]:::external
        A[API]:::api
    end
```
