# bigeyedata/terraform-modules architecture

> Architecture diagram of the GitHub repository bigeyedata/terraform-modules, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bigeyedata/terraform-modules
- Repository: https://github.com/bigeyedata/terraform-modules

## Overview

##### Project Type and Purpose
- This is an open-source Terraform Modules repository, designed to deploy the “Bigeye” analytics stack (and supporting services) into an AWS environment.
- It provides reusable modules (VPC, IAM, databases, messaging, caching, search, file systems, alarms, Bigeye application containers, etc.) plus examples for common configurations.

##### File Structure
- Top-level folders:
– `.github/workflows`: CI/CD pipelines for testing, publishing modules, validating docs and scripts
– `modules/`: each subfolder is a Terraform module mapping to one AWS resource or set of resources (e.g., `vpc`, `rds`, `redis`, `rabbitmq`, `opensearch`, `bigeye`, `alarms`, `simpleservice`, etc.)
– `examples/`: concrete usage examples of the modules in various scenarios (standard, minimal, self-managed DNS/ECR/VPC/RabbitMQ/etc.)
– `scripts/`: helper scripts for version replacement, Terraform installation in examples, image-tag updates
- `modules/bigeye` is the core module pulling together submodules (ECS tasks, CloudFront, EFS, Temporal, etc.).

##### README
- Confirms purpose: “install the Bigeye stack into an AWS Environment” with version compatibility, upgrade notes, variables reference.
- Points to `examples/standard` as a canonical deployment.

##### Core Components
a. Infrastructure-as-Code Layer
– GitHub Repository (Terraform code + workflows)
– CI/CD workflows
– Scripts for versioning and setup
b. Provisioned AWS Environment
– Networking: VPC, subnets, route tables, security groups (module: `vpc`, inline & dedicated SG rules)
– Identity & Access: IAM roles, policies (module: `admin`, `iam` in examples)
– Compute & Orchestration:
- ECS cluster (module: part of `bigeye` & `simpleservice`)
- Fargate tasks or EC2 instances
- Temporal workers (module: `bigeye/temporal.tf`)
– Data & Storage:
- RDS databases (module: `rds`)
- Redis cluster or Elasticache (module: `redis`)
- RabbitMQ (module: `rabbitmq`)
- OpenSearch domain (module: `opensearch`)
- EFS filesystem (module: `bigeye/efs.tf`)
- S3 buckets (bundles, state files, artifacts)
– Application Delivery:
- CloudFront distribution (module: `bigeye/cloudfront.tf`)
- ECR repositories (module: `bigeye` or examples/self-managed-ecr)
– Observability & Alarms:
- CloudWatch metrics, logs (agent on ECS/EC2)
- Alarms for CPU, memory, custom metrics (module: `alarms`)
c. External Dependencies
– AWS CLI / Terraform CLI / tfenv on developer/workstation
– Docker (for building images; in examples)

##### Relationships & Interactions
- GitHub workflows → validate Terraform code & push modules to registry or S3
- Terraform CLI → reads modules from local `modules/` folder, applies to AWS
- Provisioning order: networking (VPC) → IAM roles → storage & databases → messaging & cache → application containers → delivery (CloudFront) → monitoring
- Application containers communicate with databases (RDS), cache (Redis), MQ (RabbitMQ), search (OpenSearch), file system (EFS)
- CloudWatch collects logs and metrics from all services → triggers alarms

##### Architectural Patterns & Principles
- Modular design: each module encapsulates a single concern (e.g., one AWS service)
- Separation of concerns: network layer, security layer, data layer, compute layer, delivery layer, observability
- Infrastructure as code: idempotent, versioned, testable via CI workflows
- Microservices / distributed components group in ECS / Fargate

##### Critical Details to Capture
- Module dependency graph (VPC → IAM → DB/Cache/MQ → Application → Delivery → Monitoring)
- Key resource interconnections (security group peering, IAM role attachments)
- CI/CD testing & publishing steps feeding into module versions

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Infrastructure as Code Layer
    subgraph "Infrastructure as Code"
        direction TB
        GitHub["GitHub Repository"]:::infra
        CI["CI/CD Workflows"]:::infra
        Scripts["Versioning & Setup Scripts"]:::infra
        TerraformCLI["Terraform CLI / AWS CLI"]:::infra
        Docker["Docker"]:::infra
    end

    GitHub -->|triggers| CI
    CI -->|runs| Scripts
    CI -->|executes| TerraformCLI
    Docker -->|builds images| TerraformCLI

    %% Terraform Modules Layer
    subgraph "Terraform Modules"
        direction TB
        AdminIAM["module/admin (IAM)"]:::security
        VPCModule["module/vpc"]:::networking
        RDSModule["module/rds"]:::data
        RedisModule["module/redis"]:::data
        RabbitModule["module/rabbitmq"]:::data
        OpenSearchModule["module/opensearch"]:::data
        AlarmsModule["module/alarms"]:::observability
        BigeyeModule["module/bigeye (core)"]:::compute
        SimpleServiceModule["module/simpleservice"]:::compute
        ExamplesStandard["examples/standard"]:::infra
    end

    TerraformCLI -->|reads| AdminIAM
    TerraformCLI -->|reads| VPCModule
    TerraformCLI -->|reads| RDSModule
    TerraformCLI -->|reads| RedisModule
    TerraformCLI -->|reads| RabbitModule
    TerraformCLI -->|reads| OpenSearchModule
    TerraformCLI -->|reads| AlarmsModule
    TerraformCLI -->|reads| BigeyeModule
    TerraformCLI -->|reads| SimpleServiceModule
    ExamplesStandard -->|invokes| AdminIAM
    ExamplesStandard -->|invokes| VPCModule
    ExamplesStandard -->|invokes| BigeyeModule
    ExamplesStandard -->|invokes| SimpleServiceModule

    %% AWS Resources & Runtime
    subgraph "AWS Resources & Runtime"
        direction TB
        VPCRes["VPC"]:::networking
        IAMRes["IAM Roles & Policies"]:::security
        RDSRes["RDS Instance"]:::data
        RedisRes["ElastiCache Cluster"]:::data
        RabbitRes["RabbitMQ Broker"]:::data
        OpenSearchRes["OpenSearch Domain"]:::data
        ECSCluster["ECS Cluster / Fargate"]:::compute
        CloudFrontRes["CloudFront Distribution"]:::compute
        EFSRes["EFS Filesystem"]:::data
        TemporalRes["Temporal Workers"]:::compute
        CloudWatch["CloudWatch Logs & Metrics"]:::observability
        AlarmsRes["CloudWatch Alarms"]:::observability
    end

    AdminIAM -->|provisions| IAMRes
    VPCModule -->|provisions| VPCRes
    RDSModule -->|provisions| RDSRes
    RedisModule -->|provisions| RedisRes
    RabbitModule -->|provisions| RabbitRes
    OpenSearchModule -->|provisions| OpenSearchRes
    BigeyeModule -->|deploys| ECSCluster
    BigeyeModule -->|deploys| CloudFrontRes
    BigeyeModule -->|deploys| EFSRes
    BigeyeModule -->|deploys| TemporalRes
    SimpleServiceModule -->|deploys| ECSCluster

    %% Runtime Data Flows
    ECSCluster -->|reads/writes| RDSRes
    ECSCluster -->|caches| RedisRes
    ECSCluster -->|publishes/subscribes| RabbitRes
    ECSCluster -->|searches| OpenSearchRes
    ECSCluster -->|stores files on| EFSRes
    CloudFrontRes -->|serves from| ECSCluster

    %% Observability
    ECSCluster -->|logs/metrics| CloudWatch
    RDSRes -->|metrics| CloudWatch
    RedisRes -->|metrics| CloudWatch
    RabbitRes -->|metrics| CloudWatch
    OpenSearchRes -->|metrics| CloudWatch
    EFSRes -->|metrics| CloudWatch
    TemporalRes -->|metrics| CloudWatch
    CloudWatch -->|triggers| AlarmsRes

    %% Click Events
    click CI "https://github.com/bigeyedata/terraform-modules/blob/main/.github/workflows/*.yaml"
    click Scripts "https://github.com/bigeyedata/terraform-modules/blob/main/scripts/install-terraform-in-examples.sh"
    click Scripts "https://github.com/bigeyedata/terraform-modules/blob/main/scripts/replace-image-tag.sh"
    click Scripts "https://github.com/bigeyedata/terraform-modules/blob/main/scripts/replace-terraform-version.sh"
    click AdminIAM "https://github.com/bigeyedata/terraform-modules/tree/main/modules/admin/"
    click RDSModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/rds/main.tf"
    click RDSModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/rds/variables.tf"
    click RDSModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/rds/outputs.tf"
    click RedisModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/redis/main.tf"
    click RedisModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/redis/variables.tf"
    click RedisModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/redis/outputs.tf"
    click RabbitModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/rabbitmq/main.tf"
    click RabbitModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/rabbitmq/variables.tf"
    click RabbitModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/rabbitmq/outputs.tf"
    click OpenSearchModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/opensearch/main.tf"
    click OpenSearchModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/opensearch/variables.tf"
    click OpenSearchModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/opensearch/outputs.tf"
    click BigeyeModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/bigeye/main.tf"
    click BigeyeModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/bigeye/variables.tf"
    click BigeyeModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/bigeye/outputs.tf"
    click BigeyeModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/bigeye/cloudfront.tf"
    click BigeyeModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/bigeye/efs.tf"
    click BigeyeModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/bigeye/temporal.tf"
    click AlarmsModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/alarms/main.tf"
    click AlarmsModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/alarms/variables.tf"
    click AlarmsModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/alarms/ecs/main.tf"
    click AlarmsModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/alarms/elb/main.tf"
    click AlarmsModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/alarms/rds/main.tf"
    click SimpleServiceModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/simpleservice/main.tf"
    click SimpleServiceModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/simpleservice/variables.tf"
    click SimpleServiceModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/simpleservice/outputs.tf"
    click SimpleServiceModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/simpleservice/containers.tf"
    click SimpleServiceModule "https://github.com/bigeyedata/terraform-modules/blob/main/modules/simpleservice/data.tf"
    click ExamplesStandard "https://github.com/bigeyedata/terraform-modules/blob/main/examples/standard/main.tf"
    click ExamplesStandard "https://github.com/bigeyedata/terraform-modules/blob/main/examples/standard/README.md"

    %% Styles
    classDef networking fill:#ADD8E6,stroke:#000,color:#000
    classDef security fill:#87CEFA,stroke:#000,color:#000
    classDef compute fill:#98FB98,stroke:#000,color:#000
    classDef data fill:#FFDAB9,stroke:#000,color:#000
    classDef observability fill:#DDA0DD,stroke:#000,color:#000
    classDef infra fill:#F0F8FF,stroke:#000,color:#000
```
