# susin0811/es-splunk architecture

> Architecture diagram of the GitHub repository susin0811/es-splunk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/susin0811/es-splunk
- Repository: https://github.com/susin0811/es-splunk

## Overview

#### Purpose and Entry Points
This repository is an infrastructure-as-code package that deploys a distributed Splunk Enterprise 9.3 / Enterprise Security environment on AWS. Terraform is the primary control plane: `terraform/main.tf` composes the deployment, while `scripts/deploy.sh` and `scripts/deploy.ps1` wrap Terraform plan/apply/output workflows. Post-provisioning operations use `scripts/install-es-app.sh` and `scripts/health-check.sh`.

#### Root Composition and Configuration Boundary
`terraform/main.tf` wires VPC, security, S3, and Splunk compute modules into one AWS stack. Inputs—including region, EC2 key pair, Splunk artifact URI, admin password, CIDR restrictions, SmartStore, ALB, and bastion toggles—are defined in `terraform/variables.tf`; exported access endpoints and deployment metadata are in `terraform/outputs.tf`. `terraform/terraform.tfvars.example` is the operator-facing configuration template.

#### Network and Access Layer
The VPC module establishes the isolated AWS network, including subnets and NAT routing, in `terraform/modules/vpc/main.tf`. Security groups and permitted Splunk/UI/management traffic are defined in `terraform/modules/security/main.tf`. Splunk nodes are intended to remain private; a bastion and optional ALB provide controlled administrative or web entry paths.

#### Storage and Artifact Layer
`terraform/modules/s3/main.tf` creates private S3 storage for two distinct concerns: Splunk installation artifacts and SmartStore-backed data tiers. The deployment consumes an externally supplied Splunk Enterprise tarball URI, so artifact upload/prestaging is an operator responsibility. AWS IAM permissions created by the compute module allow Splunk infrastructure to access these buckets.

#### Splunk Compute Topology
`terraform/modules/splunk/main.tf` provisions the EC2/IAM/optional ALB layer for the distributed deployment:
- Search Head: Splunk Web, search processing, and Enterprise Security runtime.
- Two indexers: ingestion and replicated/searchable data storage, configured for RF=2 and SF=2.
- Cluster Manager combined with Deployment Server: indexer-cluster coordination plus app/config distribution.
- Optional bastion access and ALB exposure are infrastructure-level features of this layer.

AWS EC2, EBS, IAM, Secrets Manager, S3, VPC, and optionally ALB are material runtime dependencies.

#### Bootstrap and Cluster Formation
EC2 user-data templates install and configure Splunk during instance initialization. `terraform/modules/splunk/templates/user-data-cluster-manager.sh.tpl` establishes the cluster-management/deployment-server role; `terraform/modules/splunk/templates/user-data-indexer.sh.tpl` connects indexers to that manager and applies clustered-storage setup; `terraform/modules/splunk/templates/user-data-search-head.sh.tpl` configures the Search Head to operate against the indexer tier. Bootstrap failures are surfaced through instance-side logs rather than an application runtime in this repository.

#### Operational Lifecycle and External Boundaries
Terraform provisions only Splunk Enterprise infrastructure; Enterprise Security itself is separately licensed and supplied as an external `.spl`/`.tgz` package. `scripts/install-es-app.sh` performs the post-deployment ES installation through the deployed environment, after which operators license and configure ES in Splunk Web. `scripts/health-check.sh` validates the reachable cluster after bootstrap. AWS credentials, a preexisting EC2 key pair, Splunk binaries, licenses, and the ES package remain external prerequisites.

Main entry points: [`terraform/main.tf`](https://github.com/susin0811/es-splunk/blob/main/terraform/main.tf) (Terraform stack), [`terraform/modules/splunk/main.tf`](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/main.tf) (EC2, IAM and ingress), [`terraform/modules/s3/main.tf`](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/s3/main.tf) (Artifact and SmartStore S3), [`terraform/modules/splunk/templates/user-data-indexer.sh.tpl`](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/templates/user-data-indexer.sh.tpl) (Indexer cluster)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control["Control Plane"]
  node_tf_main["Terraform stack<br/>Terraform root<br/>[main.tf]"]
  node_tf_inputs["Deployment inputs<br/>Terraform variables<br/>[variables.tf]"]
  node_tf_outputs["Access outputs<br/>Terraform outputs<br/>[outputs.tf]"]
  node_tfvars["Operator config template<br/>tfvars example"]
  node_deploy_scripts["Deploy wrappers<br/>shell and PowerShell<br/>[deploy.sh]"]
  node_es_install["ES installation<br/>post-provision script<br/>[install-es-app.sh]"]
  node_health_check["Cluster health check<br/>post-bootstrap script<br/>[health-check.sh]"]
end

subgraph group_aws["AWS Foundation"]
  node_vpc{{"Private VPC<br/>VPC module<br/>[main.tf]"}}
  node_security{{"Security groups<br/>security module<br/>[main.tf]"}}
  node_s3[("Artifact and SmartStore S3<br/>S3 module<br/>[main.tf]")]
  node_compute["EC2, IAM and ingress<br/>Splunk compute module<br/>[main.tf]"]
end

subgraph group_splunk["Splunk Topology"]
  node_cluster_manager["Cluster Manager + DS<br/>Splunk node"]
  node_indexers[("Indexer cluster<br/>2 Splunk indexers")]
  node_search_head["Search Head"]
end

subgraph group_external["External Prerequisites"]
  node_splunk_tarball["Splunk Enterprise tarball<br/>external artifact"]
  node_aws_prereqs(("AWS credentials and key pair<br/>external prerequisite"))
  node_es_package["Enterprise Security package<br/>external licensed package"]
end

node_tfvars -->|"supplies values"| node_tf_inputs
node_deploy_scripts -->|"plan/apply"| node_tf_main
node_tf_inputs -->|"configures"| node_tf_main
node_tf_main -->|"exports"| node_tf_outputs
node_tf_main -->|"provisions"| node_vpc
node_tf_main -->|"provisions"| node_security
node_tf_main -->|"provisions"| node_s3
node_tf_main -->|"provisions"| node_compute
node_aws_prereqs -.->|"authenticates deployment"| node_deploy_scripts
node_splunk_tarball -.->|"prestaged for bootstrap"| node_s3
node_vpc -->|"private network"| node_compute
node_security -->|"traffic policy"| node_compute
node_s3 -->|"IAM-backed bucket access"| node_compute
node_compute -->|"bootstraps"| node_cluster_manager
node_compute -->|"bootstraps"| node_indexers
node_compute -->|"bootstraps"| node_search_head
node_cluster_manager -->|"forms and manages cluster"| node_indexers
node_indexers -->|"searchable data"| node_search_head
node_s3 -->|"SmartStore tier"| node_indexers
node_es_package -.->|"supplies package"| node_es_install
node_es_install -->|"installs ES"| node_search_head
node_health_check -->|"validates cluster"| node_search_head

click node_tf_main "https://github.com/susin0811/es-splunk/blob/main/terraform/main.tf"
click node_tf_inputs "https://github.com/susin0811/es-splunk/blob/main/terraform/variables.tf"
click node_tf_outputs "https://github.com/susin0811/es-splunk/blob/main/terraform/outputs.tf"
click node_tfvars "https://github.com/susin0811/es-splunk/blob/main/terraform/terraform.tfvars.example"
click node_deploy_scripts "https://github.com/susin0811/es-splunk/blob/main/scripts/deploy.sh"
click node_vpc "https://github.com/susin0811/es-splunk/blob/main/terraform/modules/vpc/main.tf"
click node_security "https://github.com/susin0811/es-splunk/blob/main/terraform/modules/security/main.tf"
click node_s3 "https://github.com/susin0811/es-splunk/blob/main/terraform/modules/s3/main.tf"
click node_compute "https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/main.tf"
click node_cluster_manager "https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/templates/user-data-cluster-manager.sh.tpl"
click node_indexers "https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/templates/user-data-indexer.sh.tpl"
click node_search_head "https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/templates/user-data-search-head.sh.tpl"
click node_es_install "https://github.com/susin0811/es-splunk/blob/main/scripts/install-es-app.sh"
click node_health_check "https://github.com/susin0811/es-splunk/blob/main/scripts/health-check.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tf_main,node_tf_inputs,node_tf_outputs,node_tfvars,node_deploy_scripts,node_es_install,node_health_check toneBlue
class node_vpc,node_security,node_s3,node_compute toneAmber
class node_cluster_manager,node_indexers,node_search_head toneMint
class node_splunk_tarball,node_aws_prereqs,node_es_package toneRose
```

## Components

### Control Plane

Terraform composition and operator workflows.

- [Terraform stack](https://github.com/susin0811/es-splunk/blob/main/terraform/main.tf): Terraform root · Composes the AWS and Splunk modules. · `terraform/main.tf`
- [Deployment inputs](https://github.com/susin0811/es-splunk/blob/main/terraform/variables.tf): Terraform variables · Defines topology, access, artifact, and feature toggles. · `terraform/variables.tf`
- [Access outputs](https://github.com/susin0811/es-splunk/blob/main/terraform/outputs.tf): Terraform outputs · Exports endpoints and deployment metadata. · `terraform/outputs.tf`
- [Operator config template](https://github.com/susin0811/es-splunk/blob/main/terraform/terraform.tfvars.example): tfvars example · `terraform/terraform.tfvars.example`
- [Deploy wrappers](https://github.com/susin0811/es-splunk/blob/main/scripts/deploy.sh): shell and PowerShell · Run Terraform plan, apply, and output workflows. · `scripts/deploy.sh`
- [ES installation](https://github.com/susin0811/es-splunk/blob/main/scripts/install-es-app.sh): post-provision script · Installs ES through the deployed environment. · `scripts/install-es-app.sh`
- [Cluster health check](https://github.com/susin0811/es-splunk/blob/main/scripts/health-check.sh): post-bootstrap script · Validates reachable cluster health. · `scripts/health-check.sh`

### AWS Foundation

Network, storage, and infrastructure access.

- [Private VPC](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/vpc/main.tf): VPC module · Creates subnets and NAT routing. · `terraform/modules/vpc/main.tf`
- [Security groups](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/security/main.tf): security module · Controls Splunk UI and management traffic. · `terraform/modules/security/main.tf`
- [Artifact and SmartStore S3](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/s3/main.tf): S3 module · Creates private buckets for binaries and data tiers. · `terraform/modules/s3/main.tf`
- [EC2, IAM and ingress](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/main.tf): Splunk compute module · Provisions EC2, EBS, IAM, optional bastion, and optional ALB. · `terraform/modules/splunk/main.tf`

### Splunk Topology

Private distributed Enterprise deployment.

- [Cluster Manager + DS](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/templates/user-data-cluster-manager.sh.tpl): Splunk node · Coordinates indexers and distributes apps/configuration. · `terraform/modules/splunk/templates/user-data-cluster-manager.sh.tpl`
- [Indexer cluster](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/templates/user-data-indexer.sh.tpl): 2 Splunk indexers · RF=2/SF=2 ingestion and searchable storage tier. · `terraform/modules/splunk/templates/user-data-indexer.sh.tpl`
- [Search Head](https://github.com/susin0811/es-splunk/blob/main/terraform/modules/splunk/templates/user-data-search-head.sh.tpl): Splunk Enterprise + ES runtime · Hosts Splunk Web and search processing. · `terraform/modules/splunk/templates/user-data-search-head.sh.tpl`

### External Prerequisites

Operator-supplied dependencies and post-deployment software.

- **Splunk Enterprise tarball**: external artifact · Operator prestages the supplied binary URI.
- **AWS credentials and key pair**: external prerequisite
- **Enterprise Security package**: external licensed package · Separately supplied and licensed .spl or .tgz package.

## Connections

- Operator config template → Deployment inputs: supplies values
- Deploy wrappers → Terraform stack: plan/apply
- Deployment inputs → Terraform stack: configures
- Terraform stack → Access outputs: exports
- Terraform stack → Private VPC: provisions
- Terraform stack → Security groups: provisions
- Terraform stack → Artifact and SmartStore S3: provisions
- Terraform stack → EC2, IAM and ingress: provisions
- AWS credentials and key pair → Deploy wrappers: authenticates deployment
- Splunk Enterprise tarball → Artifact and SmartStore S3: prestaged for bootstrap
- Private VPC → EC2, IAM and ingress: private network
- Security groups → EC2, IAM and ingress: traffic policy
- Artifact and SmartStore S3 → EC2, IAM and ingress: IAM-backed bucket access
- EC2, IAM and ingress → Cluster Manager + DS: bootstraps
- EC2, IAM and ingress → Indexer cluster: bootstraps
- EC2, IAM and ingress → Search Head: bootstraps
- Cluster Manager + DS → Indexer cluster: forms and manages cluster
- Indexer cluster → Search Head: searchable data
- Artifact and SmartStore S3 → Indexer cluster: SmartStore tier
- Enterprise Security package → ES installation: supplies package
- ES installation → Search Head: installs ES
- Cluster health check → Search Head: validates cluster