How bh10-d/ami-ubuntu-elk-packer works
Project Type and Purpose
• Project Type: Infrastructure automation / DevOps script package.
• Purpose: To build and configure an AMI (using Packer) or to set up an ELK stack directly on a local Linux machine. It installs packages (openjdk, nginx, elasticsearch, kibana, logstash, filebeat) and configures a reverse proxy (NGINX) ensuring secured access to the Kibana dashboard.
File Structure
• Top-level Files:
• README.md – Provides instructions and details about the ELK deployment.
• template.json and vars.json – Indicate the Packer configuration for building an AMI.
• Directory "script":
• Contains several shell scripts (env.sh, local_machine.sh, filebeat.yml, remove.sh, script.sh) which handle varying tasks such as environment configuration, local deployment, service manipulation, and filebeat configuration.
• The presence of these components indicates that the system design involves:
• A Packer-driven image build process.
• Shell scripts to configure and manage the ELK stack on both cloud AMI instances and local machines.
README for Additional Insights
• The README outlines two deployment modes:
• AMI creation via Packer (using template.json and vars.json).
• Direct local deployment via local_machine.sh.
• It also details the installation of several packages and the use of Nginx as a reverse proxy (with provided credentials for accessing the Kibana dashboard).
a. Main Components:
• Packer Builder: Uses template.json and vars.json to produce an AMI.
• Initialization Scripts: Under the "script" directory, which include environment setup (env.sh), service control (script.sh, remove.sh), and local deployment (local_machine.sh).
• ELK Stack Components:
• Elasticsearch – Data storage and search engine.
• Logstash – Data processing pipeline.
• Kibana – Visualization and dashboarding tool.
• Filebeat – Lightweight shipper for forwarding logs.
• Nginx Reverse Proxy: Secures and routes access to Kibana.
• External Dependencies: OpenJDK (to support ELK applications).
b. Relationships and Interactions:
• The Packer Builder initiates the AMI build process; once the image is created, instance boot-up scripts ensure services (especially Elasticsearch) are started.
• For local deployments, the local_machine.sh script orchestrates installation and configuration of all components.
• Filebeat is configured (via filebeat.yml) to ship logs to Logstash/Elasticsearch.
• Nginx acts as the intermediary for accessing Kibana, providing authentication (username: kibanaadmin, password: kibanaadmin).
c. Architectural Patterns and Design Principles:
• Separation of concerns is visible in the split between image creation for cloud instances (using Packer) and local environment deployment.
• The design leverages composition: multiple scripts each handling specific deployment tasks.
• Clear layering exists with the infrastructure provisioning (AMI creation using Packer), application installation/configuration layer (ELK components and Nginx), and runtime operations (service management scripts).
d. Relevant Technologies:
• Packer for automating machine image builds.
• Shell scripting and configuration files to manage deployments.
• Nginx as a reverse proxy.
• ELK stack components (Elasticsearch, Logstash, Kibana, Filebeat).
• OpenJDK (required by the ELK stack).
Essential Elements
• Well-labeled components representing:
• Packer builder (with template.json and vars.json as configuration inputs)
• Shell script components (grouped under “Scripts”)
• Each element of the ELK stack (Elasticsearch, Logstash, Kibana, Filebeat)
• Nginx as the reverse proxy
• The flows for AMI creation versus local deployment.
• Data flow (e.g., logs moving from Filebeat to Logstash/Elasticsearch)
• Setup/initialization sequences (image creation leading to instance boot-up)
• Dependencies (e.g., services reliant on openjdk)
• Infrastructure provisioning (Packer)
• Deployment scripts/tools (shell scripts)
• Deployed services (ELK components and Nginx)
• External dependencies (such as OpenJDK)