# yogeshojha/rengine architecture

> Architecture diagram of the GitHub repository yogeshojha/rengine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yogeshojha/rengine
- Repository: https://github.com/yogeshojha/rengine

## Overview

##### Main Components
- Web Frontend & User Interface: Represented by the “web/templates”, “web/static”, and components like “dashboard”, “targetApp”, “recon_note”, etc. This is where users interact with the system.
- Django-based Backend: The core of the application is under “web/manage.py” and the “web/reNgine” module. This component handles request processing, business logic, and API endpoints (see “web/api”).
- Scanning & Recon Modules: Separate modules like “web/scanEngine” and “web/startScan” are used for managing scan configuration, engine execution, and subscans.
- Database: Though not explicitly shown in the file tree, the README and .env usage suggest a database (likely PostgreSQL) is central for storing recon data, scan results, user credentials, and settings.
- Configuration & Orchestration: Several configuration files (like .env, docker-compose.yml, Makefile, and config/nginx files) indicate that there is an infrastructure/delivery layer, including Nginx for reverse proxy and Docker containers for the deployment environment.
- External Integrations: Features such as role-based access control, GPT integrations for vulnerability reports, external tool integrations, and notification channels (Slack/Discord/Telegram) should be indicated as “external services” or “third-party APIs.”

##### Relationships and Interactions
- Web Clients (Browsers) interact with the Frontend, which makes AJAX/API calls to the Django backend.
- The Django Web Application orchestrates the business logic – passing some requests to Celery for asynchronous scanning or report-generation tasks.
- Celery Workers pick up scanning tasks, potentially interacting with external tools and then return results to the backend.
- The backend writes/reads scan data and recon results to/from the Database.
- The reverse proxy (via Nginx configuration in the “config/nginx” directory) routes external HTTP/S requests to the web container.
- Docker Compose manages service orchestration (web, celery workers, database, etc.) with clearly defined dependencies.

##### Architectural Patterns and Design Principles
- Modular Monolithic Architecture: Although the system is built as a single Django application, the clear separation into modules (dashboard, scanEngine, targetApp, recon_note, etc.) suggests an MVC-like pattern.
- Asynchronous Processing: The integration of Celery underlines a message-driven, asynchronous task processing pattern to handle time-consuming scans and reporting.
- Containerization & Infrastructure as Code: The presence of multiple Dockerfiles, docker-compose files, and configuration scripts signifies container-based deployment and infrastructure separation.
- Extensibility & Configurability: Look for design elements in the YAML-based scan engine configurations and the loose coupling between modules (scanning, reporting, and UI).

##### Relevant Technologies and Frameworks
- Django/Python as the primary web framework and runtime environment.
- Celery for asynchronous task management.
- Nginx (with configurations in “config/nginx”) for load balancing and reverse proxy.
- Docker (and Docker Compose) for container orchestration.
- PostgreSQL (or another RDBMS) as the backend database, as indicated by environment configuration files.

##### Detail and Clarity
- Don’t hesitate to break down the web application into as many components as needed to reflect functionality (e.g., separate “Dashboard,” “API,” “Reports,” and “Scan Engine” modules).
- Workflow, particularly the scan initiation, asynchronous processing, recon data storage, and user interface visualization.
- Key deployment details (like container orchestration using Docker Compose and Nginx configuration) to provide context on how the application is set up in production vs. development.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction
    subgraph "User Interaction"
        Browser("Browser"):::external
    end

    %% Infrastructure & Orchestration
    subgraph "Infrastructure & Orchestration"
        CO("Container Orchestration"):::infra
        Env("Environment Config"):::infra
        DC1("docker-compose.yml"):::infra
        DC2("docker-compose.dev.yml"):::infra
        DC3("docker-compose.setup.yml"):::infra
        DF("Dockerfile"):::infra
        NC("Nginx Config"):::infra
        MF("Makefile & CICD"):::infra
    end

    %% Reverse Proxy
    subgraph "Reverse Proxy"
        NRP("Nginx Reverse Proxy"):::infra
    end

    %% Frontend Components
    subgraph "Frontend"
        T("Templates"):::frontend
        SA("Static Assets"):::frontend
        DB("Dashboard"):::frontend
        TA("TargetApp"):::frontend
        RN("Recon Note"):::frontend
    end

    %% Django-based Backend
    subgraph "Django Backend"
        MP("manage.py"):::backend
        RNMod("reNgine Module"):::backend
        API("API Module"):::backend
    end

    %% Scanning & Recon Modules
    subgraph "Scanning & Recon Modules"
        SE("Scan Engine"):::scanning
        SI("Scan Initiation"):::scanning
    end

    %% Asynchronous Processing
    subgraph "Asynchronous Processing"
        CC("Celery Config"):::async
        CCT("Celery Custom Task"):::async
        CE("Celery Entrypoint"):::async
        BE("Beat Entrypoint"):::async
    end

    %% Other Components
    Database("Database"):::db
    External("External Integrations"):::external

    %% Connections
    Browser -->|"HTTP_requests"| NRP
    CO -->|"orchestrates"| NRP
    CO -->|"orchestrates"| RNMod
    CO -->|"orchestrates"| CC
    CO -->|"orchestrates"| Database

    NRP -->|"routes_to"| RNMod

    RNMod -->|"serves_UI"| T
    RNMod -->|"serves_UI"| SA
    RNMod -->|"serves_UI"| DB
    RNMod -->|"serves_UI"| TA
    RNMod -->|"serves_UI"| RN

    RNMod -->|"trigger_scan"| SE
    RNMod -->|"init_scan"| SI

    RNMod -->|"submit_tasks"| CC
    RNMod -->|"submit_tasks"| CCT
    RNMod -->|"read/write"| Database
    RNMod -->|"external_API"| External

    CC -->|"store_results"| Database
    CCT -->|"store_results"| Database
    CE -->|"store_results"| Database
    BE -->|"store_results"| Database

    %% Styling Classes
    classDef frontend fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef backend fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef scanning fill:#ffeeba,stroke:#856404,stroke-width:2px;
    classDef async fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef infra fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
    classDef external fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef db fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;

    %% Click Events for Frontend Components
    click T "https://github.com/yogeshojha/rengine/tree/master/web/templates"
    click SA "https://github.com/yogeshojha/rengine/tree/master/web/static"
    click DB "https://github.com/yogeshojha/rengine/tree/master/web/dashboard"
    click TA "https://github.com/yogeshojha/rengine/tree/master/web/targetApp"
    click RN "https://github.com/yogeshojha/rengine/tree/master/web/recon_note"

    %% Click Events for Django-based Backend
    click MP "https://github.com/yogeshojha/rengine/blob/master/web/manage.py"
    click RNMod "https://github.com/yogeshojha/rengine/tree/master/web/reNgine"
    click API "https://github.com/yogeshojha/rengine/tree/master/web/api"

    %% Click Events for Scanning & Recon Modules
    click SE "https://github.com/yogeshojha/rengine/tree/master/web/scanEngine"
    click SI "https://github.com/yogeshojha/rengine/tree/master/web/startScan"

    %% Click Events for Asynchronous Processing Components
    click CC "https://github.com/yogeshojha/rengine/blob/master/web/reNgine/celery.py"
    click CCT "https://github.com/yogeshojha/rengine/blob/master/web/reNgine/celery_custom_task.py"
    click CE "https://github.com/yogeshojha/rengine/blob/master/web/celery-entrypoint.sh"
    click BE "https://github.com/yogeshojha/rengine/blob/master/web/beat-entrypoint.sh"

    %% Click Events for Infrastructure & Orchestration Components
    click Env "https://github.com/yogeshojha/rengine/blob/master/.env"
    click DC1 "https://github.com/yogeshojha/rengine/blob/master/docker-compose.yml"
    click DC2 "https://github.com/yogeshojha/rengine/blob/master/docker-compose.dev.yml"
    click DC3 "https://github.com/yogeshojha/rengine/blob/master/docker-compose.setup.yml"
    click DF "https://github.com/yogeshojha/rengine/tree/master/web/Dockerfile"
    click NC "https://github.com/yogeshojha/rengine/blob/master/config/nginx/rengine.conf"
    click MF "https://github.com/yogeshojha/rengine/tree/master/Makefile"
```
