How noaa-owp/dmod works
Main Components
a. Service Layer:
• Frontend GUI Service
• Request Service (API Gateway)
• Core Services:
• Data Service
• Scheduler Service
• Monitor Service
• Partitioner Service
• Evaluation Service
• Subset Service
b. Storage Layer:
• Redis (for service state and communication)
• MinIO (object storage)
• PostgreSQL (implied from dependencies)
c. Networks:
• Internal Network
• MPI Network
• Requests Network
┌─ Client Layer ─────────────────────────────────────┐
• Frontend GUI (TypeScript/Next.js) │
• ┬───────────────────────────┘
• (WebSocket)
┌─ Gateway Layer ───────┼───────────────────────────┐
• Request Service │ │
• ┼───────────────────────────┘
┌─ Service Layer ───────┼───────────────────────────┐
• ┌─────────┐ ┌───────┴─┐ ┌──────────┐ ┌────────┐ │
• Data │ │Scheduler│ │Evaluation│ │Monitor │ │
• Service │ │ Service │ │ Service │ │Service │ │
• ┬────┘ └────┬────┘ └────┬─────┘ └───┬────┘ │
• ┌────┴────┐ ┌────┴────┐ ┌────┴─────┐ ┌───┴────┐ │
• Partition│ │ Subset │ │ Model │ │ Other │ │
• Service │ │ Service │ │Execution │ │Services│ │
• ┘ └─────────┘ └──────────┘ └────────┘ │
• ┬───────────────────────────┘
┌─ Storage Layer ───────┼───────────────────────────┐
• ┌─────────┐ ┌───────┴─┐ ┌──────────┐ │
• Redis │ │ MinIO │ │PostgreSQL│ │
• ┘ └─────────┘ └──────────┘ │
• ┘
Network Overlay
• Internal Network (encompassing all services)
• MPI Network (for model execution)
• Requests Network (external-facing services)
Data Flow Indicators
• WebSocket connections (GUI ↔ Request Service)
• REST API calls (Data Service ↔ MinIO)
• Internal service communication
• Model execution data flow
Additional Elements
Include:
• Version/environment indicators
• Security boundaries
• Load balancing components
• Docker Swarm orchestration boundary
Specific Implementation Details
• Python-based microservices
• Docker container boundaries
• Volume mounts for model data
• Authentication flow
• Evaluation pipeline
• Model execution environment
Metadata
A section showing:
• System requirements
• Core dependencies
• Network requirements
• Storage requirements
Remember to:
• Clear labels and annotations
• Both synchronous and asynchronous communication paths
• Primary and backup paths where applicable
• Document security boundaries and protocols
• Monitoring and logging flows