# dessertlab/thorfi architecture

> Architecture diagram of the GitHub repository dessertlab/thorfi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dessertlab/thorfi
- Repository: https://github.com/dessertlab/thorfi

## Overview

##### Project type and purpose
- ThorFI is a Python-based, full-stack network fault-injection solution for OpenStack clouds.
- It consists of a front-end REST API (Controller node), per-node injection agents, a CLI client, and database utilities.
- Its goal is to let testers select virtual network resources (networks, subnets, routers, ports, floating IPs), build a fault-injection configuration, and non-intrusively inject network faults (delay, loss, corrupt, duplicate, bottleneck) into tenant virtual networks.

##### File structure
- Top-level scripts:
- thorfi_client.py: CLI that invokes front-end REST APIs.
- thorfi_frontend_agent.py (in root): front-end REST server listens on TCP port (default 7777).
- injector_agent_app/injector_agent.py: injection agent listens on TCP port (default 11223) on each Compute/Network node.
- Main packages:
- `thorfi/`: core modules for front-end logic (models, forms, network_utils, workload manager, exceptions).
- `injector_agent_app/`: REST server and utilities for injecting faults locally (ping, ssh, tc, ip namespace).
- `thorfi_db_utils/` & `utils/`: modules to create, destroy, query the ThorFI database and manage OpenStack projects/users.
- Vendored dependencies in `libs/`: OpenStack client libraries (keystoneauth1, keystoneclient, nova, neutron, glance, heat) and common utilities (oslo_utils, positional, debtcollector).
- Configuration & templates: `config.py`, YAML workload definitions (iperf.yaml, jmeter.yaml), JMeter templates.

##### README for additional insights
- The front-end agent exposes endpoints such as `/get_network_topology`, `/inject_network`, `/inject_router`, etc.
- The front-end uses Keystone for authentication, Neutron/Nova/Glance/Heat clients to discover resources and build `thorfi_item_map`.
- The injection agents receive fault configurations over TCP, then run Linux `ip netns exec ... tc qdisc` commands to apply network faults to specific NICs within namespaces.

a. Main components
- ThorFI Client (CLI)
- ThorFI Front-End Agent (REST API server)
- ThorFI Database (via thorfi_db_utils)
- OpenStack Services (Keystone, Neutron, Nova, Glance, Heat)
- Injector Agents (one per Compute/Network node)
- Linux Network Tools (ip, tc)
b. Relationships & interactions
1. Client → Front-End: HTTP(S) REST calls (port 7777)
2. Front-End → OpenStack: Keystone token, Neutron/Nova/Glance/Heat API calls to retrieve tenants, networks, ports, namespaces
3. Front-End → ThorFI DB: create/read test campaigns, store topology, retrieve injection config
4. Front-End → Injector Agents: TCP connections (port 11223) to push per-node, per-NIC fault configurations
5. Injector Agents → Linux kernel: `ip netns exec … tc qdisc add …` to inject network faults
c. Architectural patterns & design principles
- Micro-service separation: controller vs. per-node agents
- RESTful API for orchestration
- CLI client for scripting & integration
- Isolation of dependencies via vendored `libs/`
- Clear layering: API layer, orchestration layer, database layer, execution layer
d. Key technologies & libraries
- Python 2.7 scripts, PyInstaller for executables
- python-keystoneclient, python-neutronclient, python-novaclient, python-glanceclient, python-heatclient
- Linux `ip` & `tc` userland tools
- Database access via thorfi_db_utils (likely SQLAlchemy or direct SQLite/MySQL calls)

## Diagram (Mermaid)

```mermaid
graph TD
    %% Client
    subgraph "Client"
        CLI["ThorFI Client (CLI)"]:::client
    end

    %% Controller Node
    subgraph "Controller Node"
        subgraph "Front-End Agent (REST API)"
            FE["ThorFI Front-End Agent (REST API)"]:::controller
            subgraph "Front-End Core Modules"
                TL["thorfi/thorfi.py"]:::controller
                MD["thorfi/models.py"]:::controller
                FM["thorfi/forms.py"]:::controller
                NU["thorfi/network_utils.py"]:::controller
                EX["thorfi/thorfi_exceptions.py"]:::controller
                WM["thorfi/thorfi_workload.py"]:::controller
                IM["thorfi/thorfi_image_manager.py"]:::controller
            end
        end
        DB["ThorFI Database Utilities"]:::database
    end

    %% OpenStack Control Plane
    subgraph "OpenStack Control Plane"
        KS["Keystone"]:::external
        NE["Neutron"]:::external
        NO["Nova"]:::external
        GL["Glance"]:::external
        HE["Heat"]:::external
    end

    %% Compute/Network Nodes
    subgraph "Compute/Network Nodes"
        subgraph "Injector Agent"
            IA["injector_agent_app/injector_agent.py"]:::agent
            IAL["injectorAgent.py"]:::agent
            IU["injector_utils.py"]:::agent
            IUE["injector_utils_exceptions.py"]:::agent
            IPU["injector_ping_utils.py"]:::agent
            ISU["injector_ssh_utils.py"]:::agent
            NT["network_tools/tc"]:::agent
        end
    end

    %% Local Linux Tools
    subgraph "Local Linux Tools"
        LT["ip, tc"]:::external
    end

    %% Config & Templates
    subgraph "Configuration & Templates"
        CF["config.py"]:::config
        IY["iperf.yaml"]:::config
        JY["jmeter.yaml"]:::config
        JC["jmeter_conf/"]:::config
    end

    %% Vendored Libraries
    subgraph "Vendored OpenStack Client Libraries"
        LIBS["libs/"]:::external
    end

    %% Keypairs
    subgraph "Keypairs"
        K1["thorfi.key / thorfi.key.pub"]:::config
        K2["injector_agent_app/thorfi.key / thorfi.key.pub"]:::config
    end

    %% Relationships
    CLI -->|HTTP 7777| FE
    FE -->|Keystone authentication| KS
    FE -->|Neutron API calls| NE
    FE -->|Nova API calls| NO
    FE -->|Glance API calls| GL
    FE -->|Heat API calls| HE
    FE -->|DB read/write| DB
    FE -->|TCP 11223| IA
    IA -->|exec ip netns & tc qdisc| LT

    %% Click Events
    click CLI "https://github.com/dessertlab/thorfi/blob/main/thorfi_client.py"
    click FE "https://github.com/dessertlab/thorfi/blob/main/thorfi_frontend_agent.py"
    click TL "https://github.com/dessertlab/thorfi/blob/main/thorfi/thorfi.py"
    click MD "https://github.com/dessertlab/thorfi/blob/main/thorfi/models.py"
    click FM "https://github.com/dessertlab/thorfi/blob/main/thorfi/forms.py"
    click NU "https://github.com/dessertlab/thorfi/blob/main/thorfi/network_utils.py"
    click EX "https://github.com/dessertlab/thorfi/blob/main/thorfi/thorfi_exceptions.py"
    click WM "https://github.com/dessertlab/thorfi/blob/main/thorfi/thorfi_workload.py"
    click IM "https://github.com/dessertlab/thorfi/blob/main/thorfi/thorfi_image_manager.py"
    click DB "https://github.com/dessertlab/thorfi/tree/main/thorfi_db_utils/"
    click IA "https://github.com/dessertlab/thorfi/blob/main/injector_agent_app/injector_agent.py"
    click IAL "https://github.com/dessertlab/thorfi/blob/main/injector_agent_app/injectorAgent.py"
    click IU "https://github.com/dessertlab/thorfi/blob/main/injector_agent_app/injector_utils.py"
    click IUE "https://github.com/dessertlab/thorfi/blob/main/injector_agent_app/injector_utils_exceptions.py"
    click IPU "https://github.com/dessertlab/thorfi/blob/main/injector_agent_app/injector_ping_utils.py"
    click ISU "https://github.com/dessertlab/thorfi/blob/main/injector_agent_app/injector_ssh_utils.py"
    click NT "https://github.com/dessertlab/thorfi/tree/main/injector_agent_app/network_tools/tc"

    %% Styles
    classDef client fill:#B3E5FC,stroke:#0288D1,color:#000
    classDef controller fill:#C8E6C9,stroke:#2E7D32,color:#000
    classDef database fill:#FFECB3,stroke:#F9A825,color:#000
    classDef agent fill:#FFE0B2,stroke:#EF6C00,color:#000
    classDef external fill:#E1BEE7,stroke:#8E24AA,color:#000
    classDef config fill:#FFCDD2,stroke:#C62828,color:#000
```
