How bigeyedata/terraform-modules works
Project Type and Purpose
• This is an open-source Terraform Modules repository, designed to deploy the “Bigeye” analytics stack (and supporting services) into an AWS environment.
• It provides reusable modules (VPC, IAM, databases, messaging, caching, search, file systems, alarms, Bigeye application containers, etc.) plus examples for common configurations.
File Structure
• Top-level folders:
– .github/workflows: CI/CD pipelines for testing, publishing modules, validating docs and scripts
– modules/: each subfolder is a Terraform module mapping to one AWS resource or set of resources (e.g., vpc, rds, redis, rabbitmq, opensearch, bigeye, alarms, simpleservice, etc.)
– examples/: concrete usage examples of the modules in various scenarios (standard, minimal, self-managed DNS/ECR/VPC/RabbitMQ/etc.)
– scripts/: helper scripts for version replacement, Terraform installation in examples, image-tag updates
• modules/bigeye is the core module pulling together submodules (ECS tasks, CloudFront, EFS, Temporal, etc.).
README
• Confirms purpose: “install the Bigeye stack into an AWS Environment” with version compatibility, upgrade notes, variables reference.
• Points to examples/standard as a canonical deployment.
Core Components
a. Infrastructure-as-Code Layer
– GitHub Repository (Terraform code + workflows)
– CI/CD workflows
– Scripts for versioning and setup
b. Provisioned AWS Environment
– Networking: VPC, subnets, route tables, security groups (module: vpc, inline & dedicated SG rules)
– Identity & Access: IAM roles, policies (module: admin, iam in examples)
– Compute & Orchestration:
• ECS cluster (module: part of bigeye & simpleservice)
• Fargate tasks or EC2 instances
• Temporal workers (module: bigeye/temporal.tf)
– Data & Storage:
• RDS databases (module: rds)
• Redis cluster or Elasticache (module: redis)
• RabbitMQ (module: rabbitmq)
• OpenSearch domain (module: opensearch)
• EFS filesystem (module: bigeye/efs.tf)
• S3 buckets (bundles, state files, artifacts)
– Application Delivery:
• CloudFront distribution (module: bigeye/cloudfront.tf)
• ECR repositories (module: bigeye or examples/self-managed-ecr)
– Observability & Alarms:
• CloudWatch metrics, logs (agent on ECS/EC2)
• Alarms for CPU, memory, custom metrics (module: alarms)
c. External Dependencies
– AWS CLI / Terraform CLI / tfenv on developer/workstation
– Docker (for building images; in examples)
Relationships & Interactions
• GitHub workflows → validate Terraform code & push modules to registry or S3
• Terraform CLI → reads modules from local modules/ folder, applies to AWS
• Provisioning order: networking (VPC) → IAM roles → storage & databases → messaging & cache → application containers → delivery (CloudFront) → monitoring
• Application containers communicate with databases (RDS), cache (Redis), MQ (RabbitMQ), search (OpenSearch), file system (EFS)
• CloudWatch collects logs and metrics from all services → triggers alarms
Architectural Patterns & Principles
• Modular design: each module encapsulates a single concern (e.g., one AWS service)
• Separation of concerns: network layer, security layer, data layer, compute layer, delivery layer, observability
• Infrastructure as code: idempotent, versioned, testable via CI workflows
• Microservices / distributed components group in ECS / Fargate
Critical Details to Capture
• Module dependency graph (VPC → IAM → DB/Cache/MQ → Application → Delivery → Monitoring)
• Key resource interconnections (security group peering, IAM role attachments)
• CI/CD testing & publishing steps feeding into module versions