How instill-ai/instill-core works
Main Components Identification
The system consists of several key microservices that should be represented as distinct components:
a. Core Services:
• API Gateway
• Console (Frontend)
• Management Backend
• Model Backend
• Pipeline Backend
• Artifact Backend
b. Infrastructure Services:
• Database (PostgreSQL)
• Redis
• Registry
• Temporal
• OpenFGA (Authorization)
• Ray Service (ML/AI Processing)
c. Observability Stack:
• Elasticsearch
• Grafana
• InfluxDB
• Jaeger
• Prometheus
• OpenTelemetry Collector
Component Relationships and Architecture
Frontend Layer:
┌─────────────┐
• Console │
• ┘
API Gateway Layer:
┌─────────────────┐
• API Gateway │
• ┘
Service Layer:
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
• Management │ │ Model │ │ Pipeline │ │ Artifact │
• Backend │ │ Backend │ │ Backend │ │ Backend │
• ┘ └─────────────┘ └─────────────┘ └─────────────┘
Infrastructure Layer:
┌─────────┐ ┌────────┐ ┌──────────┐ ┌─────────┐ ┌────────┐
• Postgres │ │ Redis │ │ Registry │ │Temporal │ │OpenFGA │
• ┘ └────────┘ └──────────┘ └─────────┘ └────────┘
Observability Layer:
┌───────────┐ ┌─────────┐ ┌──────────┐ ┌────────┐
• Grafana │ │ Jaeger │ │InfluxDB │ │Prometheus│
• ┘ └─────────┘ └──────────┘ └────────┘
Connection Types
Different types of connections:
• Monitoring/metrics flow: Dash-dot lines
Additional Elements
Include:
• Load balancers where applicable
• Security boundaries
• External service integrations
• Volume mounts and persistent storage
• Kubernetes deployment boundaries (based on Helm charts)
Deployment Context
How the system is deployed using Kubernetes:
• Namespace boundaries
• Service mesh if applicable
• Replicated components
• Stateful vs stateless services