How akintunero/k8s-chaos-engineering works
Project Type and Purpose
• This is a Kubernetes-based Chaos Engineering demo.
• Purpose: Deploy a simple Flask web application into a K8s cluster, monitor it with Prometheus, and run chaos experiments (CPU hog, network latency, pod deletion) using LitmusChaos.
File Structure
• hello-world-app/
• Flask application source (app.py), packaged via Dockerfile.
• deployments/
• Raw Kubernetes YAML for Deployment + Service of the Flask app.
• experiments/
• LitmusChaos experiment manifests (cpu-hog, network-latency, pod-delete).
• helm/
• Helm Chart for templated deployment of the Flask app (Chart.yaml, values.yaml, deployment/service templates).
• monitoring/
• prometheus.yaml: K8s resources for Prometheus server + Service.
• .github/workflows/ci-cd.yaml
• CI/CD workflow for build, push, and helm/kubectl deploy.
Key Components
a. K8s Cluster
• Namespace(s) for: application, monitoring, chaos experiments
• Cluster control plane (API server, scheduler, etcd) – optional detail
b. Container Registry
• Docker registry where flask-chaos-demo image is pushed
c. Continuous Integration/Deployment
• GitHub Actions workflow:
• Triggers on push → Builds Docker image → Pushes to registry → Runs kubectl apply or helm install/upgrade
d. Application Layer
• Flask app Pods behind a Service (ClusterIP or LoadBalancer)
e. Observability
• Prometheus deployment & Service
• ServiceDiscovery scraping Flask app metrics endpoint
f. Chaos Engineering
• LitmusChaos Operator running in cluster
• ChaosExperiment CRs (cpu-hog, network-latency, pod-delete) targeting the Flask app pods
Relationships & Interactions
• Developer (GitHub) → CI/CD workflow → Container Registry & K8s API
• Client/User → K8s Service → Flask Pod
• Flask Pod → Exposes HTTP /metrics for Prometheus
• Prometheus → Scrapes /metrics from Flask Service
• LitmusChaos Operator → Reads ChaosExperiment CRDs → Injects faults into Flask Pods
• Monitoring & Chaos feedback loop: metrics degrade under fault injection
Architectural Patterns & Design Principles
• Infrastructure as Code: raw YAML + Helm templating
• Microservices‐style separation: app, monitoring, chaos engine in discrete namespaces
• 12-factor App: externalizing configuration via Helm values, stateless containers
• Feedback Loop: Observability informs resilience under chaos
Essential Details to Capture
• Namespace separation (e.g., default vs. monitoring vs. litmus)
• Workflow triggers and artifacts (image tag flows)
• Helm values override: indicate that the Chart/values.yaml drives deployment parameters
• Experiment scheduling: experiments applied after app + monitoring are up