How anirudh1905/chatbot works
Project Type & Purpose
• This is a full-stack microservice application—a conversational chatbot leveraging an LLM—deployed on Amazon EKS.
• The frontend is built in Python (Streamlit), and the backend hosts chat logic and LLM integrations.
• Infrastructure is entirely provisioned via Terraform, with Kubernetes manifests for runtime, and CI/CD automated through GitHub Actions.
Main Components
A. User Interface
– Browser or client interacting with Streamlit frontend.
B. Frontend Service
– src/frontend.py (Streamlit app) packaged in Dockerfile-frontend.
– Runs in a Kubernetes Deployment (“streamlit” manifest).
C. Backend Chat Service
– src/main.py hosts the chat API and handles LLM calls.
– Packaged in Dockerfile-server, runs as its own Deployment (“chatbot.yaml”).
D. LLM / Inference Engine
– Could be a hosted API (OpenAI, Hugging Face) or in-cluster model server. Represent it as an external service.
E. Kubernetes Cluster (EKS)
– Control plane and worker nodes (terraform/eks.tf).
– Namespaces, Deployments, Services, HPA (manifests/hpa.yaml), metrics-server.
F. Networking & Load Balancer
– AWS ALB Ingress Controller (terraform/AWSLoadBalancerController.json, gw.tf).
– Ingress routing user traffic to Streamlit.
G. Container Registry
– ECR repository (terraform/ecr.tf) storing both frontend & backend images.
H. Terraform-Managed AWS Infra
– VPC, subnets (vpc.tf, subnets.tf), IAM roles (iam.tf), load balancer controller, EKS add-ons.
I. CI/CD Pipelines (GitHub Actions)
– Frontend & server build & push to ECR, then kubectl apply or Helm chart upgrade (deploy_frontend.yaml, deploy_server.yaml).
J. Local Dev & Tilt
– Tiltfile for rapid iteration: simulates k8s locally, builds & deploys on file change.
Relationships & Data Flow
1. User sends a request via browser → ALB Ingress → Streamlit frontend.
2. Frontend calls backend API over internal cluster network.
3. Backend sends prompts to LLM (external API endpoint) and receives responses.
4. Backend returns chatbot response to the frontend.
5. Frontend displays the reply to the user.
6. CI/CD: On git push, GitHub Actions builds images → pushes to ECR → updates Kubernetes deployments.
7. Terraform provisions or updates AWS resources (VPC, EKS, IAM, ECR) as code.
Architectural Patterns & Design Principles
• Microservice Separation: front & back are distinct containers.
• Infrastructure-as-Code: Terraform for AWS, K8s manifests for runtime.
• GitOps/CICD: Declarative deployments via GitHub Actions.
• Dynamic Scaling: Kubernetes HPA based on CPU/memory.
• Dev-Prod Parity: Tilt for local k8s to mirror prod.
How to Lay It Out
– Streamlit Pod → Chatbot Pod → External LLM API.
– Between Pods label with “HTTP/gRPC”.
Third Row: ECR (image repo), Terraform (IaC), GitHub Actions CI/CD