# kimjanghyun1010/kubeflow_example architecture

> Architecture diagram of the GitHub repository kimjanghyun1010/kubeflow_example, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kimjanghyun1010/kubeflow_example
- Repository: https://github.com/kimjanghyun1010/kubeflow_example

## Overview

Key Components to Include in the System Design:

##### High-Level Architecture
- Iris Pipeline
- Titanic Pipeline
- Metrics Evaluation Pipeline

##### For Each Pipeline, Include These Components

a. Iris Pipeline:
- Data Loading Container (from 1_data_load)
- Model Training Container (from 2_model_training)

b. Titanic Pipeline:
- AWS S3 Storage (external component)
- Preprocessing Container
- Training Container

c. Metrics Evaluation Pipeline:
- Data Loading Container
- Model Training Container with Metrics Output
- Condition Check Component
- Metrics Visualization Component

##### Infrastructure Layer
- Kubeflow Platform
- Docker Container Registry
- Kubernetes Cluster (underlying infrastructure)

##### Labels
- Component names
- Key technologies (Python, Docker, Kubeflow)
- File formats (CSV, JSON for metrics)

##### Pipeline Orchestration
- How pipeline.py files coordinate components
- Parallel vs sequential execution

##### Data Flow Patterns
- How data moves between containers
- S3 integration in Titanic pipeline
- Metrics flow in evaluation pipeline

##### Containerization
- Docker container boundaries
- Container dependencies

##### Metrics and Evaluation
- Metrics visualization component
- Condition-based flow control
- Evaluation feedback loop

Level of Detail:

- Technical stack information in annotations
- Integration points with external services

Remember to include version information and any relevant configuration details in annotations, as these are critical for understanding the complete system architecture.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Infrastructure Layer
    subgraph Infrastructure
        KF[Kubeflow Platform]:::infra
        DCR[Docker Container Registry]:::infra
        K8S[Kubernetes Cluster]:::infra
        KF --> K8S
        DCR --> K8S
    end

    %% Iris Pipeline
    subgraph IrisPipeline[Iris Pipeline]
        IPO["Pipeline Orchestrator"]:::control
        IDL["Data Loading Container"]:::data
        IMT["Model Training Container"]:::model
        ISD["Source Data (Iris.csv)"]:::storage
        
        IPO --> IDL
        ISD --> IDL
        IDL --> IMT
    end

    %% Titanic Pipeline
    subgraph TitanicPipeline[Titanic Pipeline]
        TPO["Pipeline Orchestrator"]:::control
        S3[("AWS S3")]:::storage
        TPC["Preprocessing Container"]:::data
        TMT["Training Container"]:::model
        TSD["Source Data (train.csv)"]:::storage
        
        TPO --> TPC
        TSD --> TPC
        TPC <--> S3
        TPC --> TMT
        TMT <--> S3
    end

    %% Metrics Evaluation Pipeline
    subgraph MetricsPipeline[Metrics Evaluation Pipeline]
        MPO["Pipeline Orchestrator"]:::control
        MDL["Data Loading Container"]:::data
        MMT["Model Training Container"]:::model
        MCC{{"Condition Check"}}:::decision
        MVC["Metrics Visualization"]:::viz
        
        MPO --> MDL
        MDL --> MMT
        MMT --> MCC
        MCC -.-> MVC
    end

    %% Infrastructure Connections
    IrisPipeline --> KF
    TitanicPipeline --> KF
    MetricsPipeline --> KF

    %% Click Events
    click IPO "https://github.com/kimjanghyun1010/kubeflow_example/blob/master/iris/pipeline.py"
    click IDL "https://github.com/kimjanghyun1010/kubeflow_example/tree/master/iris/1_data_load/"
    click IMT "https://github.com/kimjanghyun1010/kubeflow_example/tree/master/iris/2_model_training/"
    click ISD "https://github.com/kimjanghyun1010/kubeflow_example/blob/master/iris/1_data_load/Iris.csv"
    
    click TPO "https://github.com/kimjanghyun1010/kubeflow_example/blob/master/titanic/pipelines.py"
    click TPC "https://github.com/kimjanghyun1010/kubeflow_example/tree/master/titanic/preprocessing/"
    click TMT "https://github.com/kimjanghyun1010/kubeflow_example/tree/master/titanic/train_model/"
    click TSD "https://github.com/kimjanghyun1010/kubeflow_example/blob/master/titanic/train.csv"
    
    click MPO "https://github.com/kimjanghyun1010/kubeflow_example/blob/master/metrics_evaluation_and_check_condition/pipeline.py"
    click MDL "https://github.com/kimjanghyun1010/kubeflow_example/tree/master/metrics_evaluation_and_check_condition/1_data_load/"
    click MMT "https://github.com/kimjanghyun1010/kubeflow_example/tree/master/metrics_evaluation_and_check_condition/2_model_training/"

    %% Styling
    classDef infra fill:#gray,stroke:#333,stroke-width:2px
    classDef control fill:#purple,stroke:#333,stroke-width:2px
    classDef data fill:#blue,stroke:#333,stroke-width:2px
    classDef model fill:#green,stroke:#333,stroke-width:2px
    classDef storage fill:#yellow,stroke:#333,stroke-width:2px
    classDef decision fill:#orange,stroke:#333,stroke-width:2px
    classDef viz fill:#pink,stroke:#333,stroke-width:2px

    %% Legend
    subgraph Legend
        L1[Infrastructure]:::infra
        L2[Control Flow]:::control
        L3[Data Processing]:::data
        L4[Model Training]:::model
        L5[Storage]:::storage
        L6{{"Decision Point"}}:::decision
        L7[Visualization]:::viz
    end
```
