# luizadevops/gcp-toolkit architecture

> Architecture diagram of the GitHub repository luizadevops/gcp-toolkit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/luizadevops/gcp-toolkit
- Repository: https://github.com/luizadevops/gcp-toolkit

## Overview

##### Project type and purpose
- This is an open-source, command-line Python toolkit (“GCP Services Toolkit”) for auditing and reporting on a Google Cloud Platform (GCP) project.
- It provides three pluggable services: a Firewall Rule Inspector/Cleaner, an IAM Policy Scanner, and a BigQuery Usage Reporter.

##### File structure
- Root files:
- main.py: CLI entry point that reads config, parses flags, and orchestrates all tools.
- config_sample.json (→ config.json): JSON configuration for project ID, thresholds, flags.
- Dockerfile / .dockerignore: containerization support.
- requirements.txt: Python dependencies.
- gcp_services_toolkit/: package containing each service as its own sub-package:
- cost_reporter/service.py
- firewall_inspector/service.py
- iam_scanner/service.py
- toolkit.py: registers and invokes each service.
- gcp_utils/: shared utility layer:
- clients.py: wraps Google API clients (Compute, IAM, BigQuery, Storage).
- utils.py: common functions (e.g., pagination, logging, error handling).

##### README for additional insights
- The three tools run by default when you execute main.py.
- Each tool:
- Invokes GCP REST/SDK APIs via gcp_utils.
- Processes & flags items based on config.json rules.
- Optionally deletes resources (--delete) or simulates (--dry-run).
- Prerequisites: Python ≥3.9, gcloud CLI, GCP APIs enabled.

a. Main components:
1) CLI Layer:
- User & flags (--config, --delete, --dry-run)
- main.py
2) Configuration:
- config.json
3) Orchestrator:
- toolkit.py (registers & dispatches each service)
4) Services (three modules):
- FirewallInspectorService
- IAMScannerService
- CostReporterService
5) Utility layer:
- GCPClientFactory (clients.py)
- SharedHelperFunctions (utils.py)
6) External Systems:
- GCP Compute API (firewall rules)
- GCP IAM & Cloud Storage APIs (bucket IAM)
- GCP BigQuery API (INFORMATION_SCHEMA query)
7) Output / Logs:
- stdout/log file, optional resource deletion.

b. Relationships & Data Flows:
1) CLI → Config Loader → Orchestrator
2) Orchestrator → each Service → gcp_utils clients → respective GCP API
3) Service receives API response → applies business rules (utils) → logs output / flags / deletes
4) All results returned to CLI → final summary printed.

c. Architectural patterns & design principles:
- Modular / Plugin-style: every tool sits in its own sub-package, and toolkit.py dynamically discovers or registers them.
- Single Responsibility: each service only handles one domain (firewall, IAM, cost).
- Layered design: CLI → Orchestrator → Service → Utilities → External APIs.
- Configuration-driven behavior: thresholds, regions, project IDs live in config.json.

d. Key technologies & frameworks:
- Python 3.9+ (type hints, Optional)
- google-cloud-bigquery, google-api-python-client
- Docker for containerization
- gcloud CLI for authentication (ADC)
- JSON for configuration

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Docker Container"
        direction TB
        subgraph "CLI Layer"
            direction TB
            Main["main.py"]:::cli
        end

        subgraph "Configuration"
            direction TB
            Config["config_sample.json"]:::config
        end

        subgraph "Orchestrator"
            direction TB
            Orchestrator["toolkit.py"]:::orchestrator
        end

        subgraph "Services" 
            direction TB
            FW["FirewallInspectorService"]:::service
            IAM["IAMScannerService"]:::service
            Cost["CostReporterService"]:::service
        end

        subgraph "Utility Layer"
            direction TB
            Clients["GCPClientFactory"]:::utility
            Utils["SharedHelperFunctions"]:::utility
        end

        subgraph "External GCP APIs"
            direction TB
            ComputeAPI["GCP Compute API"]:::external
            IAMAPI["GCP IAM & Storage API"]:::external
            BigQueryAPI["GCP BigQuery API"]:::external
        end

        subgraph "Support Files"
            direction TB
            DockerfileNode["Dockerfile"]:::support
            Dockerignore[".dockerignore"]:::support
            Requirements["requirements.txt"]:::support
        end

        subgraph "Output"
            direction TB
            Output["Output / Logs / Deletes"]:::output
        end
    end

    Main -->|"reads config & flags"| Config
    Main -->|"invokes"| Orchestrator

    Orchestrator -->|"dispatches"| FW
    Orchestrator -->|"dispatches"| IAM
    Orchestrator -->|"dispatches"| Cost

    FW -->|"uses API clients"| Clients
    IAM -->|"uses API clients"| Clients
    Cost -->|"uses API clients"| Clients

    FW -->|"uses helpers"| Utils
    IAM -->|"uses helpers"| Utils
    Cost -->|"uses helpers"| Utils

    Clients -->|"calls"| ComputeAPI
    Clients -->|"calls"| IAMAPI
    Clients -->|"calls"| BigQueryAPI

    FW -->|"outputs results"| Output
    IAM -->|"outputs results"| Output
    Cost -->|"outputs results"| Output

    click Main "https://github.com/luizadevops/gcp-toolkit/blob/main/main.py"
    click Config "https://github.com/luizadevops/gcp-toolkit/blob/main/config_sample.json"
    click Orchestrator "https://github.com/luizadevops/gcp-toolkit/blob/main/gcp_services_toolkit/toolkit.py"
    click FW "https://github.com/luizadevops/gcp-toolkit/blob/main/gcp_services_toolkit/firewall_inspector/service.py"
    click IAM "https://github.com/luizadevops/gcp-toolkit/blob/main/gcp_services_toolkit/iam_scanner/service.py"
    click Cost "https://github.com/luizadevops/gcp-toolkit/blob/main/gcp_services_toolkit/cost_reporter/service.py"
    click Clients "https://github.com/luizadevops/gcp-toolkit/blob/main/gcp_utils/clients.py"
    click Utils "https://github.com/luizadevops/gcp-toolkit/blob/main/gcp_utils/utils.py"
    click DockerfileNode "https://github.com/luizadevops/gcp-toolkit/tree/main/Dockerfile"
    click Dockerignore "https://github.com/luizadevops/gcp-toolkit/blob/main/.dockerignore"
    click Requirements "https://github.com/luizadevops/gcp-toolkit/blob/main/requirements.txt"

    classDef cli fill:#a2d2ff,stroke:#0366d6,color:#000;
    classDef config fill:#ffdfa2,stroke:#d68f03,color:#000;
    classDef orchestrator fill:#d1ffd6,stroke:#2d8a00,color:#000;
    classDef service fill:#ffd6d6,stroke:#d60000,color:#000;
    classDef utility fill:#f0d6ff,stroke:#8000a8,color:#000;
    classDef external fill:#d6f7ff,stroke:#0096a8,color:#000;
    classDef support fill:#e2e2e2,stroke:#888888,color:#000;
    classDef output fill:#fff5d6,stroke:#a86c00,color:#000;
```
