How miguelamelom/drip works
A. Project Type and Purpose
Type
• A command‐line tool and long‐running service written in Go.
• Acts as a “model serving framework” (control plane + data plane) for deploying AI/ML models.
Purpose
• Expose AI models over HTTP.
• Provide advanced release strategies (A/B testing, Canary, Shadow).
• Integrate Prometheus metrics.
• Support auto‐scaling and warm‐up.
• Declarative config via YAML/JSON.
B. File Structure Analysis
cmd/ – CLI entrypoints
• Remove, update, changeAlias, stageAlias, serve, etc.
engine/ – Business logic invoked by CLI and server
• AddModel, RemoveModel, Update, auth, raise events, stageAlias.
core/ – Runtime components for the served service
• core/modelsManager.go – registry of loaded models.
• core/serverRaiser/ – injects model servers (in‐house or MLflow).
• core/modelProxy/ – routing strategies (stable, A/B, Canary, shadow, load‐balancer).
• core/proxyStats/ – gather and expose metrics for Prometheus.
• core/warming/ – pre‐warm model endpoints.
go.mod/go.sum – dependency management.
C. README Insights
• HTTP endpoints expose models.
• Prometheus metrics baked in.
• Advanced release strategies noted.
• Configurable auto‐scaling.
• Uses declarative configs.
Control Plane (CLI/Config)
• CLI commands (cmd/) that mutate the desired state (add/remove/update models, aliases, config files).
• Config files (YAML/JSON).
Data Plane (Server)
• HTTP Server (main.go → core/server).
• Models Manager – keeps track of active model instances.
• Server Raiser – spins up model servers (inHouse or MLflow backends).
• Model Proxy – routes incoming requests per chosen strategy (stable, AB testing, Canary, shadow).
• Warming Service – pre‐loads models before routing traffic.
• Proxy Stats – collects and exposes metrics for Prometheus.
External Services
• MLflow (optional remote model store/serving).
• Prometheus (metrics ingestion).
• HTTP Clients (downstream applications invoking model endpoints).
E. Relationships & Interactions
• CLI → Engine APIs → updates config/desired state.
• Serve Command → initializes HTTP Server → Core components.
• HTTP Request → Model Proxy → forwards to one or more Model Servers (via Server Raiser).
• Warming Service → triggers Server Raiser to pre‐spin instances.
• Proxy Stats → taps into Model Proxy and Server Raiser for metrics → Prometheus.
• Server Raiser (MLflow) → calls MLflow REST API to spin up or register models.
F. Architectural Patterns & Design Principles
• Separation of Concerns:
• CLI vs. long‐running service.
• Control Plane vs Data Plane.
• Plugin/Strategy Pattern:
• serverRaiser/backends and modelProxy/strategies are extensible.
• Declarative Configuration:
• entire deployment defined in YAML/JSON.
• Observability Built-In:
• proxyStats exports Prometheus metrics.
• Modular Go Packages:
• clear package boundaries under core and engine.
G. Diagramming Guidelines
• CLI → Config files and Engine APIs.
• Serve → Core modules initialization.
• HTTP Clients → HTTP Server → Model Proxy → Model Servers.
• Model Proxy/Warming → Server Raiser.
• Core modules → Proxy Stats → Prometheus.
Detail Level
• Component with its Go package path (e.g., core/modelProxy).
• Strategy plugins under Model Proxy.
• Backend options under Server Raiser.
• Metric pull vs. push for Prometheus.
H. Tailoring to This Project
• Two operating modes:
1. CLI – mutates model state (add/update/remove/alias).
2. Serve – spins up HTTP service with core modules.
• Warming Service as a sidecar to pre‐load models.
• Both inHouse and mlflow server raisers as selectable backends.
• Configuration flow (CLI → Service).
• Data flow (Client → Proxy → Model Instances).
• Metrics flow (ProxyStats → Prometheus).
• Control plane (CLI/config) – Blue.
• Data plane (HTTP server & modules) – Green.
• External services – Gray.