# pablobascunana/youml-manager architecture

> Architecture diagram of the GitHub repository pablobascunana/youml-manager, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pablobascunana/youml-manager
- Repository: https://github.com/pablobascunana/youml-manager

## Overview

##### Main Components
- Kubernetes Integration: Inside “src/core/kubernetes” you have a jobs directory (with YAML definitions) and the jobs_manager module, which is responsible for interfacing with a Kubernetes cluster to schedule and manage ML training jobs.
- RabbitMQ Consumer: Found as a management command in “src/core/management/commands/rabbitmq_consumer.py” and supplemented by test fixtures in “src/tests/fixtures/rabbitmq_config.py”. This component processes messages/tasks asynchronously, likely to trigger or update job statuses.
- Local Development Environment Components: The project also provides Docker Compose configurations for RabbitMQ (in “.local/rabbitmq/docker-compose.yml”) and references to Minikube for running Kubernetes locally. These are part of the deployment/testing ecosystem.
- Testing Infrastructure: A dedicated “src/tests” directory contains test cases and fixtures for both the Kubernetes job management and RabbitMQ interactions.
- CI/CD Integration: The presence of GitHub workflow files (.github/workflows) for Codacy, SonarCloud, and a dependence on a Pipfile indicates that the project is designed with automated quality and dependency management in mind.

##### Relationships and Interactions
- The Django backend (server) acts as the hub that receives web requests (and possibly API calls) and coordinates operations.
- Upon triggering an operation (like starting a training job), the backend calls the Kubernetes job manager which uses Kubernetes job YAML files to submit jobs to a cluster.
- The RabbitMQ consumer runs as a command (or background process) that listens for messages, ensuring asynchronous processing. This may be used for tasks such as reporting job status or handling non-blocking operations.
- The testing suite verifies that both the Kubernetes and RabbitMQ integrations behave as expected.
- External dependencies (such as Docker for RabbitMQ and Minikube for Kubernetes) are tied into the local development environment and are clearly laid out in the README for setup.

##### Architectural Patterns and Design Principles
- Separation of Concerns: The project separates its concerns by dividing responsibilities among submodules – one for managing Kubernetes jobs, one for RabbitMQ interactions, and the standard Django layers for HTTP serving.
- Modular Integration: The Kubernetes and RabbitMQ components are loosely coupled with the main Django application via management commands and dedicated modules.
- Declarative Configuration: YAML files (in the Kubernetes jobs) and Docker Compose configurations illustrate a declarative approach to managing external systems.
- Testing and Quality Assurance: The dedicated tests and CI/CD configuration reflect an emphasis on reliability and maintainability.

## Diagram (Mermaid)

```mermaid
graph TD
    Client["User Request"]:::infra

    subgraph "Application Layer"
        Django_Main["Django Main (settings,urls,asgi,wsgi)"]:::backend
        Django_Admin["Django Admin & Core Apps (admin,apps)"]:::backend
    end

    subgraph "Kubernetes Integration"
        Kube_Manager["Kubernetes Job Manager"]:::integration
        Kube_YAML["Job YAML Definitions"]:::integration
    end

    subgraph "RabbitMQ Integration"
        RabbitMQ_Command["RabbitMQ Consumer (Management Command)"]:::integration
        RabbitMQ_Fixture["RabbitMQ Test Fixture"]:::integration
    end

    subgraph "Local Development"
        Dev_Docker["RabbitMQ Docker Compose"]:::infra
    end

    subgraph "Testing Infrastructure"
        Test_Conftest["Conftest (src/tests/conftest.py)"]:::infra
        Test_K8s["Kubernetes Jobs Test (src/tests/kubernetes/test_jobs_manager.py)"]:::infra
        Test_Rabbit["RabbitMQ Test (src/tests/test_rabbitmq_consumer.py)"]:::infra
    end

    subgraph "CI/CD Integration"
        CI_Workflows["GitHub Workflows"]:::cicd
        Dependency_Management["pipenv Dependency (Pipfile & Pipfile.lock)"]:::cicd
    end

    Client -->|"SendsRequest"| Django_Main
    Client -->|"SendsRequest"| Django_Admin
    Django_Main -->|"SubmitsJob"| Kube_Manager
    Kube_Manager -->|"UsesYAML"| Kube_YAML
    Django_Main -->|"TriggersConsumer"| RabbitMQ_Command
    Dev_Docker -->|"HostsRabbitMQ"| RabbitMQ_Command
    RabbitMQ_Command -->|"UsesFixture"| RabbitMQ_Fixture
    Test_K8s -->|"Tests"| Kube_Manager
    Test_Rabbit -->|"Tests"| RabbitMQ_Command
    CI_Workflows -->|"Monitors"| Django_Main
    CI_Workflows -->|"Monitors"| Django_Admin

    click Django_Main "https://github.com/pablobascunana/youml-manager/tree/develop/src/main"
    click Django_Admin "https://github.com/pablobascunana/youml-manager/blob/develop/src/admin.py"
    click Django_Admin "https://github.com/pablobascunana/youml-manager/blob/develop/src/core/apps.py"
    click Kube_Manager "https://github.com/pablobascunana/youml-manager/blob/develop/src/core/kubernetes/jobs_manager.py"
    click Kube_YAML "https://github.com/pablobascunana/youml-manager/blob/develop/src/core/kubernetes/jobs/train-job.yaml"
    click RabbitMQ_Command "https://github.com/pablobascunana/youml-manager/blob/develop/src/core/management/commands/rabbitmq_consumer.py"
    click RabbitMQ_Fixture "https://github.com/pablobascunana/youml-manager/blob/develop/src/tests/fixtures/rabbitmq_config.py"
    click Dev_Docker "https://github.com/pablobascunana/youml-manager/blob/develop/.local/rabbitmq/docker-compose.yml"
    click Test_Conftest "https://github.com/pablobascunana/youml-manager/blob/develop/src/tests/conftest.py"
    click Test_K8s "https://github.com/pablobascunana/youml-manager/blob/develop/src/tests/kubernetes/test_jobs_manager.py"
    click Test_Rabbit "https://github.com/pablobascunana/youml-manager/blob/develop/src/tests/test_rabbitmq_consumer.py"
    click CI_Workflows "https://github.com/pablobascunana/youml-manager/tree/develop/.github/workflows"
    click Dependency_Management "https://github.com/pablobascunana/youml-manager/tree/develop/Pipfile"

    classDef backend fill:#FFCCCB,stroke:#333,stroke-width:2px;
    classDef integration fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef infra fill:#C1E1C1,stroke:#333,stroke-width:2px;
    classDef cicd fill:#FFD700,stroke:#333,stroke-width:2px;
```
