# devopsprojects05/devsecops-end-to-end-project architecture

> Architecture diagram of the GitHub repository devopsprojects05/devsecops-end-to-end-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/devopsprojects05/devsecops-end-to-end-project
- Repository: https://github.com/devopsprojects05/devsecops-end-to-end-project

## Overview

##### Project type and purpose
- This is an End-to-End DevSecOps reference implementation. It bundles:
- A small Node.js web application
- A Jenkins-centric CI/CD pipeline
- Infrastructure-as-Code (Terraform)
- Configuration management (Ansible)
- Security tooling (SonarQube, Trivy, TFSec, Snyk)
- Secrets management (HashiCorp Vault)
- Artifact management (Nexus)
- Observability & monitoring (OpenTelemetry Collector, Prometheus, Grafana)

##### File structure
- Top-level:
- Jenkinsfile/Jenkinsfile – pipeline definition
- terraform/ – sets up EC2 and other cloud resources
- ansible/ – playbook to configure or deploy the app
- src/ – Node.js app, Dockerfile, OpenTelemetry config, test suite
- Prometheus/prometheus.yaml – monitoring scrape config
- Patterns:
- “src” contains the application (MVC-style Node.js server.js + public/)
- “terraform” for provisioning
- “ansible” for post-provisioning configuration
- A single EC2 instance running Jenkins plus multiple tool containers

##### Key insights from the README
- All DevSecOps tools are co-hosted on the Jenkins server to cut costs
- The pipeline:
1. Checks out code
2. Builds and tests (Jest, SonarQube, Snyk)
3. Lints and scans infrastructure (TFSec)
4. Builds Docker image and scans with Trivy
5. Pushes image to Nexus or Docker registry
6. Provisions infra via Terraform
7. Deploys application via Ansible
8. Monitors runtime with Prometheus/Grafana and traces via OpenTelemetry

a. Main components
- CI/CD Server (Jenkins)
- Source Control (GitHub)
- Build & Test Tools (Jest, SonarQube, Snyk)
- IaC Provisioning (Terraform)
- Configuration Management (Ansible)
- Container Registry (Nexus or Docker Hub)
- Secrets Manager (Vault)
- Application (Node.js) in Docker
- Observability: OTel Collector, Prometheus, Grafana
- Notification channel (Slack)
b. Relationships & interactions
- Developer commits → GitHub → Jenkins pipeline triggers
- Jenkins orchestrates:
- SonarQube scanner
- Snyk & Jest tests
- Terraform plan/apply
- Docker build & Trivy scan → push to Nexus
- Ansible deployment to target VM/container
- Node.js app emits traces/metrics → OTel Collector → Prometheus
- Scanner outputs (Trivy, TFSec) → custom metrics → Prometheus
- Grafana queries Prometheus → dashboards
- Alerts from Grafana → Slack
- Jenkins and Ansible pull secrets from Vault via plugin
c. Architectural patterns & design principles
- Pipeline Orchestration: Jenkins as control plane
- Infrastructure as Code: Terraform + TFSec validation
- Immutable artifacts: Docker image baked once, scanned, versioned in Nexus
- Configuration as Code: Ansible playbooks
- Centralized observability: OTel → Prometheus → Grafana
- Single-server cost optimization: co-hosting tools but logically separated by containers
d. Key technologies
- Jenkins (CI/CD), Docker (containers), Node.js/Express, Terraform (IaC), Ansible, HashiCorp Vault, SonarQube & Scanner, Snyk, TFSec, Trivy, Nexus Repository, OpenTelemetry Collector, Prometheus, Grafana

##### Detail & separation
- Scanner feeding outputs into Prometheus (via exporter scripts)
- Service ports (e.g., Jenkins:8080, SonarQube:9000, Grafana:3001)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layer 1: Developer & SCM
    subgraph "Layer 1: Developer & SCM"
        direction TB
        Developer["Developer"]:::external
        GitHub["GitHub (SCM)"]:::external
    end

    Developer -->|commits code| GitHub
    GitHub -->|webhook| Jenkins

    %% Layer 2: CI/CD & Orchestration
    subgraph "Layer 2: CI/CD & Orchestration"
        direction TB
        Jenkins["Jenkins (CI/CD)"]:::ci
        Checkout["Checkout Stage"]:::ci
        BuildTest["Build & Test\n(Jest, SonarQube, Snyk)"]:::ci
        IaCScan["IaC Scan\n(TFSec)"]:::security
        DockerBuild["Docker Build & Trivy Scan"]:::security
        PushArtifact["Push to Nexus"]:::storage
    end

    Jenkins --> Checkout
    Jenkins --> BuildTest
    Jenkins --> IaCScan
    Jenkins --> DockerBuild
    DockerBuild --> PushArtifact

    %% Layer 4: Artifact & Secrets
    subgraph "Layer 4: Artifact & Secrets"
        direction TB
        Nexus["Nexus Repository"]:::storage
        Vault["Vault (Secrets)"]:::storage
    end

    PushArtifact --> Nexus
    Jenkins -->|retrieve secrets| Vault
    Ansible -->|retrieve secrets| Vault

    %% Layer 5: Provisioning & Deployment
    subgraph "Layer 5: Provisioning & Deployment"
        direction TB
        Terraform["Terraform"]:::infra
        Ansible["Ansible"]:::infra
    end

    Jenkins --> Terraform
    Jenkins --> Ansible
    Terraform -->|provision infra| VM["Target VM/Container"]:::external
    Ansible -->|deploy app| VM

    %% Layer 6: Runtime & Observability
    subgraph "Layer 6: Runtime & Observability"
        direction TB
        App["Node.js App\n(in Docker)"]:::app
        OTel["OpenTelemetry Collector"]:::monitor
        Prometheus["Prometheus TSDB"]:::monitor
        Grafana["Grafana"]:::monitor
    end

    VM --> App
    App -->|metrics & traces| OTel
    OTel --> Prometheus
    Prometheus --> Grafana

    %% Layer 7: Notifications
    subgraph "Layer 7: Notification"
        direction TB
        Slack["Slack"]:::external
    end

    Grafana -->|alerts| Slack

    %% CI/CD triggers provisioning & deployment
    Checkout --> BuildTest
    BuildTest --> IaCScan
    IaCScan --> DockerBuild
    DockerBuild --> PushArtifact
    PushArtifact --> Terraform
    Terraform --> Ansible

    %% Click Events
    click Jenkinsfile "https://github.com/devopsprojects05/devsecops-end-to-end-project/tree/main/Jenkinsfile"
    click Checkout "https://github.com/devopsprojects05/devsecops-end-to-end-project/tree/main/Jenkinsfile/Jenkinsfile"
    click Terraform "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/terraform/main.tf"
    click Terraform "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/terraform/variables.tf"
    click Terraform "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/terraform/output.tf"
    click Ansible "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/ansible/playbook.yml"
    click App "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/src/server.js"
    click App "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/src/package.json"
    click App "https://github.com/devopsprojects05/devsecops-end-to-end-project/tree/main/src/public/"
    click App "https://github.com/devopsprojects05/devsecops-end-to-end-project/tree/main/src/Dockerfile"
    click App "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/src/.dockerignore"
    click BuildTest "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/src/jest.config.js"
    click BuildTest "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/src/test/server.test.js"
    click BuildTest "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/src/test/jest.config.js"
    click OTel "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/src/otel-collector-config.yaml"
    click Prometheus "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/Prometheus/prometheus.yaml"
    click Images "https://github.com/devopsprojects05/devsecops-end-to-end-project/tree/main/Images/"
    click Readme "https://github.com/devopsprojects05/devsecops-end-to-end-project/blob/main/Readme.md"

    %% Styles
    classDef external fill:#f9f,stroke:#333,stroke-width:1px
    classDef ci fill:#add8e6,stroke:#333,stroke-width:1px
    classDef security fill:#f88,stroke:#333,stroke-width:1px
    classDef storage fill:#ffa500,stroke:#333,stroke-width:1px
    classDef infra fill:#d3d3d3,stroke:#333,stroke-width:1px
    classDef app fill:#87ceeb,stroke:#333,stroke-width:1px
    classDef monitor fill:#90ee90,stroke:#333,stroke-width:1px
```
