How itselavia/edge-computing-platform-for-deep-learning-apps works
Main Components
a. Frontend:
• Located in the “frontend” directory (notably in the “my-app” subfolder) and includes a React-based application with various UI components (e.g., Login, Dashboard, Devices, etc.).
• Containerized with Docker (Dockerfile present) and managed via docker-compose.
b. Backend Microservices:
• Two key subprojects under “backend”:
• “flask_service”: A Flask-based service that appears to handle inference requests or service logic (including deployment configurations, credentials, and custom inference files).
• “model_manager”: A Go-based service that is likely responsible for managing machine learning models, deploying them onto the cluster (as seen by its own deployment scripts and Dockerfile).
• Both backends are containerized and include Kubernetes deployment manifests (YAML files in deploy folders).
c. Database and Storage:
• Scripts for database creation (“db_creation.sql” under backend/DB_scripts) indicate a MySQL (or similar) backend database.
d. Infrastructure and Deployment:
• Terraform files under the “infra” directory, with several modules (e.g., VPC, Kubernetes, KubeEdge, GPU node, tflite converter function), show that the infrastructure is provisioned as code.
• The “docker-compose.yml” at the root and in the “frontend/my-app” directory provide local orchestration guidelines.
e. Use Cases and Examples:
• The “use_cases” directory includes example implementations for TensorRT object detection and object detection using TFLite models.
f. Miscellaneous and Auxiliary Files:
• The “example” folder contains helper scripts (e.g., create_config.sh, inference.py) along with YAML files for Kubernetes configuration, which further illustrate how the platform can be engaged and tested.
g. External Services and Cloud APIs:
• The README details interactions with GCP services (Cloud Functions, Cloud Build, Storage, Virtual Machine, VPC, IAM, Cloud SQL) as well as Azure for GPU worker nodes.
• Curl commands in the README show endpoints for model conversion, deployment, and user management, indicating API-based interactions over the network.
Relationships and Interactions
• Users interact with the Frontend (React) application which communicates with the backend services via API calls.
• The Flask service likely handles data processing/inference and interacts directly with the model repository (contained in inferencefiles and models) and possibly with the database.
• The Model Manager (written in Go) administers the deployment of models – managing replicas, handling GPU support and scaling on Kubernetes.
• Both backend services communicate with a MySQL database (accessed via scripts in the backend/DB_scripts) and might interact with external Cloud APIs and cloud storage.
• Infrastructure is managed via Terraform which orchestrates setting up Kubernetes clusters (worker and edge nodes), VPC and other cloud networking components.
• Docker is used extensively for containerization, with Dockerfiles present in the “flask_service”, “model_manager”, “frontend/my-app”, and in the “infra/tflite_inference_base_image” directories.
Architectural Patterns & Design Principles
• Microservices Architecture: The separation of model management and inference into distinct backend components.
• Containerization and Orchestration: Use of Docker and Kubernetes manifests indicate that the system is built with cloud-native, containerized microservices in mind.
• Infrastructure as Code: Use of Terraform modules to provision and manage cloud resources.
• CI/CD principles: Makefile rules (e.g., cluster-init, cluster-deploy) point toward automation of deployment and integration testing.
• Separation of Concerns: Distinct directories for frontend UI, business logic in backends, infrastructure management, and use-case examples.