How kimjanghyun1010/kubeflow_example works
Key Components to Include in the System Design:
High-Level Architecture
• Iris Pipeline
• Titanic Pipeline
• Metrics Evaluation Pipeline
For Each Pipeline, Include These Components
a. Iris Pipeline:
• Data Loading Container (from 1_data_load)
• Model Training Container (from 2_model_training)
b. Titanic Pipeline:
• AWS S3 Storage (external component)
• Preprocessing Container
• Training Container
c. Metrics Evaluation Pipeline:
• Data Loading Container
• Model Training Container with Metrics Output
• Condition Check Component
• Metrics Visualization Component
Infrastructure Layer
• Kubeflow Platform
• Docker Container Registry
• Kubernetes Cluster (underlying infrastructure)
Labels
• Component names
• Key technologies (Python, Docker, Kubeflow)
• File formats (CSV, JSON for metrics)
Pipeline Orchestration
• How pipeline.py files coordinate components
• Parallel vs sequential execution
Data Flow Patterns
• How data moves between containers
• S3 integration in Titanic pipeline
• Metrics flow in evaluation pipeline
Containerization
• Docker container boundaries
• Container dependencies
Metrics and Evaluation
• Metrics visualization component
• Condition-based flow control
• Evaluation feedback loop
Level of Detail:
• Technical stack information in annotations
• Integration points with external services
Remember to include version information and any relevant configuration details in annotations, as these are critical for understanding the complete system architecture.