# noaa-owp/dmod architecture

> Architecture diagram of the GitHub repository noaa-owp/dmod, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/noaa-owp/dmod
- Repository: https://github.com/noaa-owp/dmod

## Overview

##### Main Components

a. Service Layer:
- Frontend GUI Service
- Request Service (API Gateway)
- Core Services:
- Data Service
- Scheduler Service
- Monitor Service
- Partitioner Service
- Evaluation Service
- Subset Service

b. Storage Layer:
- Redis (for service state and communication)
- MinIO (object storage)
- PostgreSQL (implied from dependencies)

c. Networks:
- Internal Network
- MPI Network
- Requests Network

**┌─ Client Layer ─────────────────────────────────────┐**
- Frontend GUI (TypeScript/Next.js)                  │
- ┬───────────────────────────┘
- (WebSocket)
**┌─ Gateway Layer ───────┼───────────────────────────┐**
- Request Service       │                           │
- ┼───────────────────────────┘

**┌─ Service Layer ───────┼───────────────────────────┐**
- ┌─────────┐ ┌───────┴─┐ ┌──────────┐ ┌────────┐ │
- Data   │ │Scheduler│ │Evaluation│ │Monitor │ │
- Service │ │ Service │ │ Service  │ │Service │ │
- ┬────┘ └────┬────┘ └────┬─────┘ └───┬────┘ │

- ┌────┴────┐ ┌────┴────┐ ┌────┴─────┐ ┌───┴────┐ │
- Partition│ │ Subset  │ │  Model   │ │ Other  │ │
- Service │ │ Service │ │Execution │ │Services│ │
- ┘ └─────────┘ └──────────┘ └────────┘ │
- ┬───────────────────────────┘

**┌─ Storage Layer ───────┼───────────────────────────┐**
- ┌─────────┐ ┌───────┴─┐ ┌──────────┐           │
- Redis  │ │  MinIO  │ │PostgreSQL│           │
- ┘ └─────────┘ └──────────┘           │
- ┘

##### Network Overlay

- Internal Network (encompassing all services)
- MPI Network (for model execution)
- Requests Network (external-facing services)

##### Data Flow Indicators

- WebSocket connections (GUI ↔ Request Service)
- REST API calls (Data Service ↔ MinIO)
- Internal service communication
- Model execution data flow

##### Additional Elements

Include:
- Version/environment indicators
- Security boundaries
- Load balancing components
- Docker Swarm orchestration boundary

##### Specific Implementation Details

- Python-based microservices
- Docker container boundaries
- Volume mounts for model data
- Authentication flow
- Evaluation pipeline
- Model execution environment

##### Metadata

A section showing:
- System requirements
- Core dependencies
- Network requirements
- Storage requirements

Remember to:
- Clear labels and annotations
- Both synchronous and asynchronous communication paths
- Primary and backup paths where applicable
- Document security boundaries and protocols
- Monitoring and logging flows

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles and Classes
    classDef frontend fill:#e1f5fe,stroke:#01579b
    classDef gateway fill:#fff3e0,stroke:#ff6f00
    classDef service fill:#e8f5e9,stroke:#2e7d32
    classDef storage fill:#fce4ec,stroke:#880e4f
    classDef network fill:#f3e5f5,stroke:#4a148c,stroke-dasharray: 5 5
    
    %% Client Layer
    subgraph ClientLayer["Client Layer"]
        GUI["Frontend GUI Service<br/>(dmod-gui)"]:::frontend
    end
    
    %% Gateway Layer
    subgraph GatewayLayer["Gateway Layer"]
        RequestService["Request Service<br/>(request-service)"]:::gateway
    end
    
    %% Service Layer
    subgraph ServiceLayer["Service Layer"]
        DataService["Data Service<br/>(data-service)"]:::service
        SchedulerService["Scheduler Service<br/>(scheduler-service)"]:::service
        MonitorService["Monitor Service<br/>(monitor-service)"]:::service
        PartitionerService["Partitioner Service<br/>(partitioner-service)"]:::service
        EvaluationService["Evaluation Service<br/>(evaluation-service)"]:::service
        SubsetService["Subset Service<br/>(subset-service)"]:::service
        ModelExecution["Model Execution<br/>(model-exec)"]:::service
    end
    
    %% Storage Layer
    subgraph StorageLayer["Storage Layer"]
        Redis["Redis<br/>(myredis)"]:::storage
        MinIO["MinIO<br/>(minio)"]:::storage
        PostgreSQL["PostgreSQL<br/>(postgres)"]:::storage
    end
    
    %% Network Boundaries
    subgraph InternalNetwork["Internal Network"]
        ServiceLayer
        StorageLayer
    end
    
    %% Connections
    GUI <--> |WebSocket| RequestService
    RequestService --> DataService
    RequestService --> SchedulerService
    RequestService --> MonitorService
    
    DataService <--> MinIO
    DataService <--> Redis
    DataService <--> PostgreSQL
    
    SchedulerService --> ModelExecution
    SchedulerService <--> Redis
    
    MonitorService --> ModelExecution
    MonitorService <--> Redis
    
    PartitionerService <--> Redis
    EvaluationService <--> Redis
    SubsetService <--> Redis
    
    %% Click Events
    click GUI "https://github.com/noaa-owp/dmod/tree/master/ts/dmod_gui"
    click RequestService "https://github.com/noaa-owp/dmod/tree/master/python/services/requestservice"
    click DataService "https://github.com/noaa-owp/dmod/tree/master/python/services/dataservice"
    click SchedulerService "https://github.com/noaa-owp/dmod/tree/master/python/services/schedulerservice"
    click MonitorService "https://github.com/noaa-owp/dmod/tree/master/python/services/monitorservice"
    click PartitionerService "https://github.com/noaa-owp/dmod/tree/master/python/services/partitionerservice"
    click EvaluationService "https://github.com/noaa-owp/dmod/tree/master/python/services/evaluationservice"
    click SubsetService "https://github.com/noaa-owp/dmod/tree/master/python/services/subsetservice"
    click Redis "https://github.com/noaa-owp/dmod/tree/master/docker/main/myredis"
    click MinIO "https://github.com/noaa-owp/dmod/tree/master/docker/object_store"
    
    %% Legend
    subgraph Legend
        Frontend["Frontend Component"]:::frontend
        Gateway["Gateway Component"]:::gateway
        ServiceComp["Service Component"]:::service
        StorageComp["Storage Component"]:::storage
    end
```
