# bastientlc/k8s-video-conversion architecture

> Architecture diagram of the GitHub repository bastientlc/k8s-video-conversion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bastientlc/k8s-video-conversion
- Repository: https://github.com/bastientlc/k8s-video-conversion

## Overview

##### Main Components
- Frontend – Under the “src/frontend” directory. This is the client-side application (built with Vite) used for interacting with end users.
- API – Under “src/flask_api”. This is the backend REST API (using Flask) that handles video uploads and receives user requests.
- Worker – Under “src/ffmpeg_worker”. This component consumes video processing jobs from a Kafka topic and processes them (likely using FFmpeg) before storing or sending the results.
- Kafka – Deployed as part of the Kubernetes setup (files in the “kubernetes/kafka” folder) with defined topics for task and result queues.
- MinIO – Hosted via Kubernetes (in the “kubernetes/minio” folder) to act as shared storage for storing processed video outputs.
- Kubernetes Components – Various YAML files across “kubernetes/app”, “kubernetes/prometheus”, “kubernetes/dashboards”, and also supporting deployment files for Kafka, monitoring, and load testing.
- Docker Compose Setup – Provided at the root (docker-compose.yaml) which can run these services locally.

##### Relationships and Interactions
- The end-user interacts with the system primarily through the Frontend.
- The Frontend communicates with the API (HTTP calls) for video uploads.
- Upon receiving a video upload, the API places a “task job” message on a Kafka topic.
- The Worker subscribes to the Kafka task queue, processes the video, and then posts the resulting message (or signals completion) via a result queue.
- Processed videos or metadata might then be stored into MinIO.
- Auto-scaling and orchestration are managed by Kubernetes (with Horizontal Pod Autoscalers (HPA) and, for workers, possibly KEDA).

##### Architectural Patterns and Design Principles
- Microservices Architecture – The separation into distinct services (Frontend, API, Worker) that communicate over messaging (Kafka) aligns with a microservices pattern.
- Event-Driven Communication – Use of Kafka topics (between API and worker components) supports asynchronous processing.
- Containerization and Orchestration – Docker and Kubernetes configuration files indicate deployment via containerized microservices.
- Scalability – Use of Kubernetes auto-scaling (HPA and KEDA) and separation of concerns in the system design.
- Observability – Integration of Prometheus and Grafana for system monitoring and performance analysis.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Subgraph: User Interaction
    subgraph "User Interaction"
        user("User"):::userStyle
    end

    %% Subgraph: Client Side
    subgraph "Client Side"
        fe["Frontend (src/frontend)"]:::ui
    end

    %% Subgraph: Backend Processing
    subgraph "Backend Processing"
        api["API (src/flask_api)"]:::backend
        worker["Worker (src/ffmpeg_worker)"]:::backend
    end

    %% Subgraph: Messaging & Storage
    subgraph "Messaging & Storage"
        kafka["Kafka"]:::messaging
        minio["MinIO Storage"]:::storage
    end

    %% Subgraph: Infrastructure & Monitoring
    subgraph "Infrastructure & Monitoring"
        k8s["Kubernetes Orchestration"]:::infra
        mon["Monitoring & Logging (Prometheus, Grafana)"]:::monitoring
        load["Load Testing (Locust)"]:::testing
        dcompose["Local Development (Docker Compose)"]:::dev
    end

    %% Connections: User and Client Side
    user -->|"HTTP_Request"| fe

    %% Connections: Client Side to Backend
    fe -->|"HTTP_API"| api

    %% Connections: API to Messaging
    api -->|"Publish_Task"| kafka

    %% Connections: Messaging to Worker
    kafka -->|"Consume_Task"| worker

    %% Connections: Worker interactions
    worker -->|"Publish_Result"| kafka
    worker -->|"Store_Output"| minio

    %% Metrics reporting to Monitoring
    fe -->|"Metrics"| mon
    api -->|"Metrics"| mon
    worker -->|"Metrics"| mon
    kafka -->|"Metrics"| mon

    %% Deployment connections with Kubernetes
    fe -->|"Deployed_on"| k8s
    api -->|"Deployed_on"| k8s
    worker -->|"Deployed_on"| k8s
    kafka -->|"Deployed_on"| k8s
    minio -->|"Deployed_on"| k8s
    mon -->|"Deployed_on"| k8s
    load -->|"Deployed_on"| k8s

    %% Local development reference
    dcompose -.->|"LocalSetup"| fe

    %% Click Events for Component Mapping
    click fe "https://github.com/bastientlc/k8s-video-conversion/tree/main/src/frontend"
    click api "https://github.com/bastientlc/k8s-video-conversion/tree/main/src/flask_api"
    click worker "https://github.com/bastientlc/k8s-video-conversion/tree/main/src/ffmpeg_worker"
    click kafka "https://github.com/bastientlc/k8s-video-conversion/tree/main/kubernetes/kafka"
    click minio "https://github.com/bastientlc/k8s-video-conversion/tree/main/kubernetes/minio"
    click k8s "https://github.com/bastientlc/k8s-video-conversion/tree/main/kubernetes/app"
    click mon "https://github.com/bastientlc/k8s-video-conversion/tree/main/kubernetes/prometheus"
    click dcompose "https://github.com/bastientlc/k8s-video-conversion/blob/main/docker-compose.yaml"
    click load "https://github.com/bastientlc/k8s-video-conversion/tree/main/kubernetes/locust"

    %% Styles
    classDef ui fill:#C6E2FF,stroke:#333,stroke-width:2px;
    classDef backend fill:#FFD700,stroke:#333,stroke-width:2px;
    classDef messaging fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef storage fill:#FFA07A,stroke:#333,stroke-width:2px;
    classDef infra fill:#D3D3D3,stroke:#333,stroke-width:2px;
    classDef monitoring fill:#FFB6C1,stroke:#333,stroke-width:2px;
    classDef dev fill:#E6E6FA,stroke:#333,stroke-width:2px;
    classDef testing fill:#F0E68C,stroke:#333,stroke-width:2px;
    classDef userStyle fill:#AFEEEE,stroke:#333,stroke-width:2px;
```
