# capitansec/portpilot architecture

> Architecture diagram of the GitHub repository capitansec/portpilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/capitansec/portpilot
- Repository: https://github.com/capitansec/portpilot

## Overview

##### Main Components
- API Component (Directory: Api):
– Built with FastAPI, this is the core backend service.
– It contains configuration files (for database, Elasticsearch, Redis), HTTP routes (for scan and user-related endpoints), middleware for authentication, and various providers for logging, cryptography, environment handling, and interfacing with RabbitMQ queues.
– The API acts as the entry point: it accepts scan requests, validates user credentials, and forwards scan jobs into the messaging middleware.
- Frontend/Panel Component (Directory: Panel):
– A React-based web UI (as indicated by the package.json and various React source files).
– Provides dashboards, login and registration forms, and components like scan viewers and tables.
– It consumes API endpoints from the backend and offers real-time visualizations (supported by Grafana for scan results visualization).
- Worker Component (Directory: Worker):
– Implements the scanning agents. The Worker retrieves scan tasks from a RabbitMQ queue (configured via QueueContext and QueueConsume) and executes scanning routines (located in Worker/Scanner).
– Once a scan is executed, the results are sent to Elasticsearch (handled via ElasticContext) for further storage and analysis.

##### External and Integrated Services
- Messaging Middleware: RabbitMQ – Acts as the central job queue where scan requests are published by the API and consumed by multiple Worker agents.
- Data Stores:
– Elasticsearch – Stores and indexes scan results.
– Database(s) – Likely PostgreSQL (and possibly MongoDB based on the badges mentioned in the README) are used for storing user data and application metadata.
– Redis – Used for caching or session management, as configured in the API.
- Other Tools:
– Jenkins (see Jenkinsfile) for CI/CD integration.
– Docker and Docker Compose (multiple Dockerfiles and docker-compose files) for containerized deployments.
– Grafana – For visualizing and monitoring the scan results analysis.

##### Architectural Patterns and Relationships
- The API acts as the intermediary, receiving requests (both from the Panel UI and external API calls) and orchestrating scan processes.
- The messaging pattern is central: API submits tasks to RabbitMQ, the distributed Worker agents compete to pop and process these tasks, and finally results get stored in Elasticsearch.
- The system follows a distributed microservices style, with clear separation between the user interface (Panel), the API (as an orchestrator/service layer), and backend processing (Worker).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Panel (React UI) Subgraph
    subgraph "Panel (React UI)"
        P_Pkg["package.json"]:::panel
        P_PkgLock["package-lock.json"]:::panel
        P_Index["index.html"]:::panel
        P_Manifest["manifest.json"]:::panel
        P_Robots["robots.txt"]:::panel
        P_App["App.js"]:::panel
        P_ScanViewer["ScanViewer.js"]:::panel
        P_ScanTable["ScanTable.js"]:::panel
        Panel_API["Panel_calls_API"]:::panel
    end

    %% API (FastAPI) Subgraph
    subgraph "API (FastAPI)"
        %% Configurations
        API_Elastic["ElasticContext.py"]:::api
        API_Database["database_connector.py"]:::api
        API_Redis["redis_connector.py"]:::api
        %% HTTP Routes
        API_Scan["scan.py"]:::api
        API_User["user.py"]:::api
        %% Middleware
        API_Auth["Auth_Token_Middleware"]:::api
        %% Providers
        API_Log["Log_Writer"]:::api
        API_Crypto["SHA256"]:::api
        API_Env["Environment_Provider"]:::api
        API_Queue["Queue_Context"]:::api
    end

    %% Worker (Scanning Agent) Subgraph
    subgraph "Worker (Scanning Agent)"
        Worker_Elastic["ElasticContext.py"]:::worker
        Worker_QueueConsume["QueueConsume.py"]:::worker
        Worker_Queue["QueueContext.py"]:::worker
        Worker_Scan["scan.py"]:::worker
        Worker_Main["main.py"]:::worker
    end

    %% Messaging Middleware
    subgraph "Messaging Middleware"
        RabbitMQ["RabbitMQ"]:::messaging
    end

    %% Data Stores Subgraph
    subgraph "Data Stores"
        Elasticsearch["Elasticsearch"]:::datastore
        Database["Database"]:::datastore
        Redis["Redis"]:::datastore
    end

    %% External Tools Subgraph
    subgraph "External Tools"
        Grafana["Grafana"]:::external
        Jenkins["CI/CD_(Jenkins)"]:::external
    end

    %% Deployment Subgraph
    subgraph "Deployment"
        Docker_Api["Api/Dockerfile"]:::deployment
        Docker_Panel["Panel/Dockerfile"]:::deployment
        Docker_Worker["Worker/Dockerfile"]:::deployment
        Compose1["docker-compose.yaml"]:::deployment
        Compose2["docker-compose.dev.yaml"]:::deployment
    end

    %% Connections between Panel and API
    P_App ---|"triggers"| Panel_API
    Panel_API ---|"REST_CALL"| API_Queue

    %% API internal connections (for illustration)
    API_Scan ---|"uses_DB"| API_Database
    API_User ---|"uses_DB"| API_Database
    API_Auth ---|"secures"| API_Scan
    API_Log --- API_Auth
    API_Crypto --- API_Env

    %% API to Messaging
    API_Queue ---|"enqueue_job"| RabbitMQ

    %% Messaging to Worker
    RabbitMQ ---|"dispatch_task"| Worker_QueueConsume

    %% Worker internal flow
    Worker_Queue --- Worker_QueueConsume
    Worker_QueueConsume ---|"initiate_scan"| Worker_Scan
    Worker_Scan ---|"result_to"| Worker_Elastic
    Worker_Scan --- Worker_Main

    %% API Data store interactions
    API_Database ---|"stores_user_data"| Database
    API_Redis ---|"cache"| Redis
    API_Elastic ---|"elastic_config"| Elasticsearch

    %% Worker storing scan results
    Worker_Scan ---|"store_results"| Elasticsearch

    %% Data visualization from Elasticsearch
    Elasticsearch ---|"visualize"| Grafana

    %% Deployment links (illustrative connections)
    Docker_Api --- API_Elastic
    Docker_Panel --- P_Pkg
    Docker_Worker --- Worker_Main
    Compose1 --- Docker_Api
    Compose1 --- Docker_Panel
    Compose1 --- Docker_Worker
    Jenkins --- Compose1
    Jenkins --- Compose2

    %% Click Events for API Component
    click API_Elastic "https://github.com/capitansec/portpilot/blob/main/Api/Config/ElasticContext.py"
    click API_Database "https://github.com/capitansec/portpilot/blob/main/Api/Config/database_connector.py"
    click API_Redis "https://github.com/capitansec/portpilot/blob/main/Api/Config/redis_connector.py"
    click API_Scan "https://github.com/capitansec/portpilot/blob/main/Api/Http/Routes/scan.py"
    click API_User "https://github.com/capitansec/portpilot/blob/main/Api/Http/Routes/user.py"
    click API_Auth "https://github.com/capitansec/portpilot/blob/main/Api/Middleware/Auth/token.py"
    click API_Log "https://github.com/capitansec/portpilot/blob/main/Api/Providers/Log/log_writter.py"
    click API_Crypto "https://github.com/capitansec/portpilot/blob/main/Api/Providers/Cryptology/sha256.py"
    click API_Env "https://github.com/capitansec/portpilot/blob/main/Api/Providers/Environment/environment.py"
    click API_Queue "https://github.com/capitansec/portpilot/blob/main/Api/Providers/Queue/QueueContext.py"

    %% Click Events for Frontend / Panel Component
    click P_Pkg "https://github.com/capitansec/portpilot/blob/main/Panel/package.json"
    click P_PkgLock "https://github.com/capitansec/portpilot/blob/main/Panel/package-lock.json"
    click P_Index "https://github.com/capitansec/portpilot/blob/main/Panel/public/index.html"
    click P_Manifest "https://github.com/capitansec/portpilot/blob/main/Panel/public/manifest.json"
    click P_Robots "https://github.com/capitansec/portpilot/blob/main/Panel/public/robots.txt"
    click P_App "https://github.com/capitansec/portpilot/blob/main/Panel/src/App.js"
    click P_ScanViewer "https://github.com/capitansec/portpilot/blob/main/Panel/src/components/dashboard/ScanViewer.js"
    click P_ScanTable "https://github.com/capitansec/portpilot/blob/main/Panel/src/components/dashboard/ScanTable.js"

    %% Click Events for Worker Component
    click Worker_Elastic "https://github.com/capitansec/portpilot/blob/main/Worker/Config/ElasticContext.py"
    click Worker_QueueConsume "https://github.com/capitansec/portpilot/blob/main/Worker/Config/QueueConsume.py"
    click Worker_Queue "https://github.com/capitansec/portpilot/blob/main/Worker/Config/QueueContext.py"
    click Worker_Scan "https://github.com/capitansec/portpilot/blob/main/Worker/Scanner/scan.py"
    click Worker_Main "https://github.com/capitansec/portpilot/blob/main/Worker/main.py"

    %% Click Events for Deployment and External Tools
    click Docker_Api "https://github.com/capitansec/portpilot/tree/main/Api/Dockerfile"
    click Docker_Panel "https://github.com/capitansec/portpilot/tree/main/Panel/Dockerfile"
    click Docker_Worker "https://github.com/capitansec/portpilot/tree/main/Worker/Dockerfile"
    click Compose1 "https://github.com/capitansec/portpilot/blob/main/docker-compose.yaml"
    click Compose2 "https://github.com/capitansec/portpilot/blob/main/docker-compose.dev.yaml"
    click Jenkins "https://github.com/capitansec/portpilot/tree/main/Jenkinsfile"

    %% Styles
    classDef api fill:#FFDDCC,stroke:#D2691E,stroke-width:2px;
    classDef panel fill:#CCFFCC,stroke:#228B22,stroke-width:2px;
    classDef worker fill:#CCE5FF,stroke:#1E90FF,stroke-width:2px;
    classDef messaging fill:#FFFFCC,stroke:#DAA520,stroke-width:2px;
    classDef datastore fill:#FFCCCC,stroke:#B22222,stroke-width:2px;
    classDef external fill:#E6E6FA,stroke:#8A2BE2,stroke-width:2px;
    classDef deployment fill:#F0FFF0,stroke:#2E8B57,stroke-width:2px;
```
