# drghassen/pfe-mpssr architecture

> Architecture diagram of the GitHub repository drghassen/pfe-mpssr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/drghassen/pfe-mpssr
- Repository: https://github.com/drghassen/pfe-mpssr

## Overview

1. **Repo purpose**
   CloudSentinel is a DevSecOps framework for Azure that combines:
   - **shift-left** preventive scanning and policy checks,
   - **shift-right** drift detection and runtime remediation,
   - **OPA** for decision logic,
   - **DefectDojo** as the findings/exceptions backend,
   - **Azure Terraform infrastructure** for the demo/deployment environment.

2. **Top-level architecture**
   The repository is organized around three main planes:
   - `shift-left/`: source, container, and IaC scanning before deployment.
   - `shift-right/`: continuous drift/prowler/custodian enforcement after deployment.
   - `policies/` + `config/`: shared decision and exception definitions used by both planes.

3. **Shift-left subsystem**
   This is the pre-deployment security pipeline. It includes scanners and normalizers for:
   - **Checkov** custom Azure policy catalog,
   - **Trivy** filesystem/config/image scanning,
   - **Gitleaks** secret scanning,
   - **Cloud-init scanner** for malicious initialization patterns,
   - **OPA** gate decisions,
   - report normalization and upload to DefectDojo.
   The scripts under `ci/scripts/shift-left/` orchestrate these steps in CI.

4. **Shift-right subsystem**
   This is the runtime control loop for deployed Azure resources. It includes:
   - **drift-engine**: detects configuration drift and builds remediation/detection reports,
   - **Prowler** Azure checks for security posture,
   - **Custodian** policies for enforcement/remediation,
   - exception fetching and gate enforcement logic,
   - uploads of drift/prowler results to DefectDojo.
   The scripts under `ci/scripts/shift-right/` connect the components into the runtime pipeline.

5. **Policy and decision layer**
   The `policies/opa/` tree is the central policy engine:
   - `gate/`: pre-deploy approvals/denials and intent-based gating,
   - `drift/`: drift classification, severity, exceptions, and remediation reasoning,
   - `prowler/`: security finding evaluation and severity actions,
   - `system/`: authorization rules for policy access.
   Tests are colocated as `.rego` test files. These policies form the architecture’s “decision boundary” between raw scanner output and actionable security outcomes.

6. **Exception and contract model**
   Exceptions are first-class artifacts, not ad hoc overrides:
   - `config/opa/data/exceptions.json` and `drift_exceptions.json` feed policy evaluation,
   - `ci/contracts/*.json` define artifact and integrity contracts,
   - `shift-left/opa/schema/exceptions_v2.schema.json` and other schemas validate structure,
   - `verification/` contains runtime-state validation for shift-right integrity.
   This means findings, exceptions, and remediation state are governed by explicit contracts.

7. **Report normalization and data flow**
   Raw scanner outputs are normalized into CloudSentinel-specific schemas before policy evaluation or storage:
   - `shift-left/normalizer/` converts scanner results into a common report format,
   - `shift-right/drift-engine/utils/json_normalizer.py` and related modules normalize drift/OPA outputs,
   - schemas under `shift-left/normalizer/schema/` and `shift-right/drift-engine/schemas/` define the canonical shapes.
   The architecture depends on normalization to keep scanners interchangeable.

8. **Infrastructure boundary**
   `infra/azure/` provides Terraform modules for the Azure environment:
   - networking, identity, key vault, storage, database/cosmos, compute, container instances, RBAC, recovery, resource groups.
   The compute module includes cloud-init templates, which ties infrastructure provisioning to the shift-left cloud-init scanner and the runtime drift model.
   This is the main cloud boundary where the security framework is exercised.

9. **Containerized toolchain**
   `ci/images/` defines dedicated Docker images for repeatable CI/runtime execution:
   - `opa`, `prowler-tools`, `scan-tools`, `deploy-tools`, `custodian-tools`, `aci-runtime`.
   These images isolate scanner dependencies and make the CI pipeline deterministic. The GitLab CI files at the root orchestrate building and using those images.

10. **Local integration and support tooling**
   `docker-compose.yml`, `demosecrets/`, and `scripts/` support local demos and operational tasks:
   - fetching exceptions from DefectDojo,
   - creating risk acceptance artifacts,
   - audit/notification utilities,
   - bootstrap scripts for OPA auth and CA setup.
   These are support boundaries around the core scanning/remediation flows.

11. **Security taxonomy and mapping**
   Several directories exist to map findings into a unified security vocabulary:
   - `shift-right/drift-engine/utils/security_taxonomy.py`,
   - `shift-left/normalizer/cs_norm_mappings.py`,
   - `config/remediation-capabilities.json`,
   - `ci/scripts/shift-right/correlation_mappings.json`.
   This indicates the architecture is designed to correlate different tools’ findings to common control, severity, and remediation categories.

12. **Main end-to-end flows**
   - **Shift-left flow:** source/IaC/artifacts → scanner execution → normalization → OPA decisioning → contract checks / DefectDojo upload.
   - **Shift-right flow:** live Azure state → drift/prowler/custodian evaluation → OPA decisions and exception matching → gate/remediation/enforcement → DefectDojo/reporting.
   The architecture centers on these two loops, with OPA and normalized schemas acting as the integration backbone.

13. **Operational emphasis**
   The presence of CI pipelines, build scripts, schemas, tests, and integrity contracts shows this repo is not just policy code; it is a full operational framework with strong emphasis on reproducibility, policy governance, and traceable security state.

Main entry points: [`ci/scripts/shift-left`](https://github.com/drghassen/pfe-mpssr/tree/main/ci/scripts/shift-left) (CI scripts), [`ci/scripts/shift-right`](https://github.com/drghassen/pfe-mpssr/tree/main/ci/scripts/shift-right) (Shift-right OPA), [`shift-left/opa`](https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/opa) (Shift-left OPA), [`infra/azure`](https://github.com/drghassen/pfe-mpssr/tree/main/infra/azure) (Azure infra)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_shift_left["Shift-left"]
  node_n_shift_left_ci["CI scripts<br/>orchestrator"]
  node_n_checkov["Checkov<br/>iac scanner"]
  node_n_trivy["Trivy<br/>container scanner"]
  node_n_gitleaks["Gitleaks<br/>secret scanner"]
  node_n_cloudinit["Cloud-init scan<br/>pattern scanner"]
  node_n_normalizer["Normalizer<br/>schema mapper"]
  node_n_shiftleft_opa["Shift-left OPA<br/>decision gate"]
end

subgraph group_g_shift_right["Shift-right"]
  node_n_shift_right_ci["Runtime scripts<br/>orchestrator"]
  node_n_drift_engine["Drift engine<br/>drift pipeline"]
  node_n_prowler["Prowler<br/>posture scanner"]
  node_n_custodian["Custodian<br/>remediator"]
  node_n_shiftright_opa["Shift-right OPA<br/>decision engine"]
end

subgraph group_g_policies["Policy layer"]
  node_n_policy_gate["Gate policies<br/>rego policy"]
  node_n_policy_drift["Drift policies<br/>rego policy"]
  node_n_policy_prowler["Prowler policies<br/>rego policy"]
  node_n_policy_authz["System authz<br/>rego policy"]
  node_n_exceptions[("Exceptions<br/>shared data")]
end

subgraph group_g_platform["Platform"]
  node_n_contracts["Contracts<br/>artifact contract"]
  node_n_verification["Verification<br/>integrity check"]
  node_n_azure_infra["Azure infra<br/>terraform"]
  node_n_images["Tool images<br/>container images"]
end

node_n_shift_left_ci -->|"runs"| node_n_checkov
node_n_shift_left_ci -->|"runs"| node_n_trivy
node_n_shift_left_ci -->|"runs"| node_n_gitleaks
node_n_shift_left_ci -->|"runs"| node_n_cloudinit
node_n_shift_left_ci -->|"normalizes"| node_n_normalizer
node_n_normalizer -->|"feeds"| node_n_shiftleft_opa
node_n_shiftleft_opa -->|"evaluates"| node_n_policy_gate
node_n_shiftleft_opa -->|"loads"| node_n_exceptions
node_n_shift_left_ci -->|"checks"| node_n_contracts
node_n_shift_left_ci -->|"uses"| node_n_images
node_n_shift_right_ci -->|"runs"| node_n_drift_engine
node_n_shift_right_ci -->|"runs"| node_n_prowler
node_n_shift_right_ci -->|"runs"| node_n_custodian
node_n_shift_right_ci -->|"gates"| node_n_shiftright_opa
node_n_drift_engine -->|"feeds"| node_n_shiftright_opa
node_n_prowler -->|"feeds"| node_n_shiftright_opa
node_n_shiftright_opa -->|"evaluates"| node_n_policy_drift
node_n_shiftright_opa -->|"evaluates"| node_n_policy_prowler
node_n_shiftright_opa -->|"loads"| node_n_exceptions
node_n_custodian -->|"remediates"| node_n_azure_infra
node_n_azure_infra -->|"defines"| node_n_cloudinit
node_n_azure_infra -->|"observed by"| node_n_drift_engine
node_n_contracts -->|"supports"| node_n_verification
node_n_policy_authz -->|"authorizes"| node_n_shiftleft_opa
node_n_policy_authz -->|"authorizes"| node_n_shiftright_opa

click node_n_shift_left_ci "https://github.com/drghassen/pfe-mpssr/tree/main/ci/scripts/shift-left"
click node_n_checkov "https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/checkov"
click node_n_trivy "https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/trivy"
click node_n_gitleaks "https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/gitleaks"
click node_n_cloudinit "https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/cloudinit-scanner"
click node_n_normalizer "https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/normalizer"
click node_n_shiftleft_opa "https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/opa"
click node_n_shift_right_ci "https://github.com/drghassen/pfe-mpssr/tree/main/ci/scripts/shift-right"
click node_n_drift_engine "https://github.com/drghassen/pfe-mpssr/tree/main/shift-right/drift-engine"
click node_n_prowler "https://github.com/drghassen/pfe-mpssr/tree/main/shift-right/prowler"
click node_n_custodian "https://github.com/drghassen/pfe-mpssr/tree/main/shift-right/custodian"
click node_n_shiftright_opa "https://github.com/drghassen/pfe-mpssr/tree/main/ci/scripts/shift-right"
click node_n_policy_gate "https://github.com/drghassen/pfe-mpssr/tree/main/policies/opa/gate"
click node_n_policy_drift "https://github.com/drghassen/pfe-mpssr/tree/main/policies/opa/drift"
click node_n_policy_prowler "https://github.com/drghassen/pfe-mpssr/tree/main/policies/opa/prowler"
click node_n_policy_authz "https://github.com/drghassen/pfe-mpssr/tree/main/policies/opa/system"
click node_n_exceptions "https://github.com/drghassen/pfe-mpssr/tree/main/config/opa/data"
click node_n_contracts "https://github.com/drghassen/pfe-mpssr/tree/main/ci/contracts"
click node_n_verification "https://github.com/drghassen/pfe-mpssr/tree/main/verification"
click node_n_azure_infra "https://github.com/drghassen/pfe-mpssr/tree/main/infra/azure"
click node_n_images "https://github.com/drghassen/pfe-mpssr/tree/main/ci/images"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_shift_left_ci,node_n_checkov,node_n_trivy,node_n_gitleaks,node_n_cloudinit,node_n_normalizer,node_n_shiftleft_opa toneBlue
class node_n_shift_right_ci,node_n_drift_engine,node_n_prowler,node_n_custodian,node_n_shiftright_opa toneAmber
class node_n_policy_gate,node_n_policy_drift,node_n_policy_prowler,node_n_policy_authz,node_n_exceptions toneMint
class node_n_contracts,node_n_verification,node_n_azure_infra,node_n_images toneRose
```

## Components

### Shift-left

Pre-deployment scanning, normalization, policy gating, and report upload.

- [CI scripts](https://github.com/drghassen/pfe-mpssr/tree/main/ci/scripts/shift-left): orchestrator · Shift-left CI entrypoints that sequence scanners, normalization, OPA checks, and DefectDojo upload. · `ci/scripts/shift-left`
- [Checkov](https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/checkov): iac scanner · Custom Azure Checkov policy catalog for infrastructure checks. · `shift-left/checkov`
- [Trivy](https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/trivy): container scanner · Filesystem, config, and image scanning with Azure-focused mappings. · `shift-left/trivy`
- [Gitleaks](https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/gitleaks): secret scanner · Secret scanning and suppression handling for source and configs. · `shift-left/gitleaks`
- [Cloud-init scan](https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/cloudinit-scanner): pattern scanner · Detects malicious cloud-init and initialization patterns tied to compute templates. · `shift-left/cloudinit-scanner`
- [Normalizer](https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/normalizer): schema mapper · Converts scanner outputs into a common CloudSentinel report schema. · `shift-left/normalizer`
- [Shift-left OPA](https://github.com/drghassen/pfe-mpssr/tree/main/shift-left/opa): decision gate · Fetches exceptions and evaluates pre-deploy gate decisions. · `shift-left/opa`

### Shift-right

Post-deployment drift detection, posture checks, remediation, and reporting.

- [Runtime scripts](https://github.com/drghassen/pfe-mpssr/tree/main/ci/scripts/shift-right): orchestrator · Shift-right pipeline entrypoints for drift, posture, enforcement, and reporting. · `ci/scripts/shift-right`
- [Drift engine](https://github.com/drghassen/pfe-mpssr/tree/main/shift-right/drift-engine): drift pipeline · Detects drift, normalizes state, enriches findings, and builds reports. · `shift-right/drift-engine`
- [Prowler](https://github.com/drghassen/pfe-mpssr/tree/main/shift-right/prowler): posture scanner · Azure security posture detection for runtime findings. · `shift-right/prowler`
- [Custodian](https://github.com/drghassen/pfe-mpssr/tree/main/shift-right/custodian): remediator · Policy-based enforcement and autofix for Azure resources. · `shift-right/custodian`
- [Shift-right OPA](https://github.com/drghassen/pfe-mpssr/tree/main/ci/scripts/shift-right): decision engine · Evaluates drift and prowler findings against exceptions and actions. · `ci/scripts/shift-right`

### Policy layer

OPA decision logic and shared exception/config inputs.

- [Gate policies](https://github.com/drghassen/pfe-mpssr/tree/main/policies/opa/gate): rego policy · Pre-deploy approval, deny, and warning logic. · `policies/opa/gate`
- [Drift policies](https://github.com/drghassen/pfe-mpssr/tree/main/policies/opa/drift): rego policy · Drift classification, severity, exception matching, and remediation reasoning. · `policies/opa/drift`
- [Prowler policies](https://github.com/drghassen/pfe-mpssr/tree/main/policies/opa/prowler): rego policy · Evaluates posture findings and maps them to actions and severity. · `policies/opa/prowler`
- [System authz](https://github.com/drghassen/pfe-mpssr/tree/main/policies/opa/system): rego policy · Authorization boundary for policy access and execution. · `policies/opa/system`
- [Exceptions](https://github.com/drghassen/pfe-mpssr/tree/main/config/opa/data): shared data · Canonical exception and remediation inputs consumed by both planes. · `config/opa/data`

### Platform

Azure infrastructure, container images, and contract/verification support.

- [Contracts](https://github.com/drghassen/pfe-mpssr/tree/main/ci/contracts): artifact contract · Artifact and integrity contracts for CI and runtime data exchange. · `ci/contracts`
- [Verification](https://github.com/drghassen/pfe-mpssr/tree/main/verification): integrity check · Runtime-state validation for shift-right integrity. · `verification`
- [Azure infra](https://github.com/drghassen/pfe-mpssr/tree/main/infra/azure): terraform · Terraform-managed Azure environment exercised by the security framework. · `infra/azure`
- [Tool images](https://github.com/drghassen/pfe-mpssr/tree/main/ci/images): container images · Deterministic CI/runtime images for scanners, policy, deployment, and remediation tools. · `ci/images`

## Connections

- CI scripts → Checkov: runs · Orchestrates infrastructure policy checks.
- CI scripts → Trivy: runs · Triggers filesystem/config/image scans.
- CI scripts → Gitleaks: runs · Triggers secret scanning.
- CI scripts → Cloud-init scan: runs · Triggers cloud-init pattern analysis.
- CI scripts → Normalizer: normalizes · Converts raw outputs to the common report schema.
- Normalizer → Shift-left OPA: feeds · Provides normalized findings for gate evaluation.
- Shift-left OPA → Gate policies: evaluates · Uses gate rules to approve, deny, or warn.
- Shift-left OPA → Exceptions: loads · Consumes exception data for gate decisions.
- CI scripts → Contracts: checks · Validates artifact and integrity contracts in CI.
- CI scripts → Tool images: uses · Runs inside dedicated scanner/tool images.
- Runtime scripts → Drift engine: runs · Coordinates drift detection and report generation.
- Runtime scripts → Prowler: runs · Triggers Azure posture checks.
- Runtime scripts → Custodian: runs · Executes enforcement and remediation policies.
- Runtime scripts → Shift-right OPA: gates · Evaluates runtime findings and gate conditions.
- Drift engine → Shift-right OPA: feeds · Provides normalized drift findings and context.
- Prowler → Shift-right OPA: feeds · Provides security posture findings for evaluation.
- Shift-right OPA → Drift policies: evaluates · Uses drift rules for classification and remediation reasoning.
- Shift-right OPA → Prowler policies: evaluates · Uses prowler rules for severity and action mapping.
- Shift-right OPA → Exceptions: loads · Consumes shared exceptions for runtime gating.
- Custodian → Azure infra: remediates · Enforces resource-state changes in Azure.
- Azure infra → Cloud-init scan: defines · Compute templates include cloud-init content that is scanned pre-deploy.
- Azure infra → Drift engine: observed by · Live Azure state is compared against desired state and plans.
- Contracts → Verification: supports · Contracts define the runtime integrity checks and expected artifacts.
- System authz → Shift-left OPA: authorizes · Controls access to policy evaluation paths.
- System authz → Shift-right OPA: authorizes · Controls access to runtime policy evaluation paths.