How bastientlc/k8s-video-conversion works
Main Components
• Frontend – Under the “src/frontend” directory. This is the client-side application (built with Vite) used for interacting with end users.
• API – Under “src/flask_api”. This is the backend REST API (using Flask) that handles video uploads and receives user requests.
• Worker – Under “src/ffmpeg_worker”. This component consumes video processing jobs from a Kafka topic and processes them (likely using FFmpeg) before storing or sending the results.
• Kafka – Deployed as part of the Kubernetes setup (files in the “kubernetes/kafka” folder) with defined topics for task and result queues.
• MinIO – Hosted via Kubernetes (in the “kubernetes/minio” folder) to act as shared storage for storing processed video outputs.
• Kubernetes Components – Various YAML files across “kubernetes/app”, “kubernetes/prometheus”, “kubernetes/dashboards”, and also supporting deployment files for Kafka, monitoring, and load testing.
• Docker Compose Setup – Provided at the root (docker-compose.yaml) which can run these services locally.
Relationships and Interactions
• The end-user interacts with the system primarily through the Frontend.
• The Frontend communicates with the API (HTTP calls) for video uploads.
• Upon receiving a video upload, the API places a “task job” message on a Kafka topic.
• The Worker subscribes to the Kafka task queue, processes the video, and then posts the resulting message (or signals completion) via a result queue.
• Processed videos or metadata might then be stored into MinIO.
• Auto-scaling and orchestration are managed by Kubernetes (with Horizontal Pod Autoscalers (HPA) and, for workers, possibly KEDA).
Architectural Patterns and Design Principles
• Microservices Architecture – The separation into distinct services (Frontend, API, Worker) that communicate over messaging (Kafka) aligns with a microservices pattern.
• Event-Driven Communication – Use of Kafka topics (between API and worker components) supports asynchronous processing.
• Containerization and Orchestration – Docker and Kubernetes configuration files indicate deployment via containerized microservices.
• Scalability – Use of Kubernetes auto-scaling (HPA and KEDA) and separation of concerns in the system design.
• Observability – Integration of Prometheus and Grafana for system monitoring and performance analysis.