# ahmadhatahet/hil-fault-test-gen architecture

> Architecture diagram of the GitHub repository ahmadhatahet/hil-fault-test-gen, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ahmadhatahet/hil-fault-test-gen
- Repository: https://github.com/ahmadhatahet/hil-fault-test-gen

## Overview

##### Project Type and Purpose
- This is a research‐oriented LLM-powered fault test‐case generation and evaluation tool for real-time fault injection in automotive software systems.
- It ingests functional safety requirements (Excel files), uses prompt templates + LLM APIs to generate fault injection test cases (TCs), runs those TCs on a hardware‐in‐the‐loop (HIL) platform, and analyzes the outcomes.

##### Main Components
a. Data Layer
- Directory: `data/`
- Contents: Excel spreadsheets of actuator/sensor requirements and examples
- Role: Source of functional safety requirements (FSRs) and ground‐truth examples
b. Configuration Layer
- Files: `.env.template` → `.env`, `pyproject.toml`, `uv.lock`
- Role: Defines environment variables (API keys, model endpoints), Python dependencies, and the project manager (“uv”) setup
c. Core Library (“TestGen”)
- Package: `testgen/`
- Modules:
– `prompts/` (prompt templates for sensors, actuators, system/user prompts, bulk vs. single)
– `analysis_utils.py`, `utils.py` (data loading, formatting, metrics)
- Role: Encapsulates LLM interaction logic, prompt management, and output parsing
d. Orchestration Layer (Notebooks & Scripts)
- Files: `*.ipynb` (e.g., `experiment_bulk.ipynb`, `experiment_all_single.ipynb`, `analyze_actuators.ipynb`, `results_analysis.ipynb`)
- Role: Runs experiments in various modes (single, bulk, dropped, actuator+sensor), invokes TestGen functions, collects results, visualizes data
e. External Services
- LLM Providers:
– OpenAI (gpt-4o, gpt-4o-mini), Ollama (phi4), local Llama models, Qwen2.5
– Configured via `.env`
- HIL System
– A high-fidelity automotive simulator / hardware-in-the-loop rig (external to repo)
- Role: Execute generated fault injection test cases in real time
f. Output & Reporting Layer
- Directories:
– `results/` (JSON files of TC execution outcomes and accuracy metrics)
– `tables/` (LaTeX tables summarizing stats)
– `figures/` (plots generated by notebooks)
- Role: Persist raw and aggregated results for analysis and publication

##### Relationships and Data Flows
1. Data Ingestion
[Data Layer] → loads Excel FSRs/examples → feeds
2. Test Case Generation
[Orchestration (Notebook)] calls → [TestGen Library]
- Passes in FSRs + prompt template
- TestGen calls → [LLM API Service]
- Receives generated fault TCs
3. Real-Time Execution
Generated TCs → sent to → [External HIL System]
HIL returns execution logs/results → saved in → `results/`
4. Analysis & Reporting
[Analysis Notebooks] load `results/` → generate metrics → output to `tables/` and `figures/`

##### Architectural Patterns & Principles
- Pipeline Pattern: linear stages (ingest → generate → execute → analyze)
- Modular Separation: core library (`testgen`) decoupled from orchestration (`.ipynb`)
- Pluggable Backends: multiple LLM providers via injectable configuration (.env)
- Reproducibility: environment and dependencies locked via `uv`, `pyproject.toml`, `uv.lock`

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data & Configuration Layer
    subgraph "Data Layer"
        D["Data Layer (data/)"]:::data
    end

    subgraph "Configuration Layer"
        ENV[".env.template"]:::config
        TOML["pyproject.toml"]:::config
        LOCK["uv.lock"]:::config
        REQ["requirements.in"]:::config
    end

    %% Processing Layer
    subgraph "Orchestration Layer"
        direction TB
        EXP1["experiment_bulk.ipynb"]:::internal
        EXP2["experiment_bulk_ollama.ipynb"]:::internal
        EXP3["experiment_all_single.ipynb"]:::internal
        EXP4["experiment_all_single_ollama.ipynb"]:::internal
        EXP5["experiment_all_single_drop_sensor.ipynb"]:::internal
        EXP6["experiment_all_single_drop_sensor_ollama.ipynb"]:::internal
        EXP7["experiment_actuator_sensor.ipynb"]:::internal
        EXP8["experiment_actuator_sensor_ollama.ipynb"]:::internal
        EXP9["analyze_actuators.ipynb"]:::internal
        EXP10["chat_analysis.ipynb"]:::internal
        EXP11["results_analysis.ipynb"]:::internal
    end

    subgraph "TestGen Library (testgen/)"
        direction TB
        TGInit["__init__.py"]:::internal
        AU["analysis_utils.py"]:::internal
        U["utils.py"]:::internal
        subgraph "prompts/"
            P1["Sensors.py"]:::internal
            P2["SensorActuator.py"]:::internal
            P3["SystemPrompt.py"]:::internal
            P4["UserPrompt.py"]:::internal
            P5["UserPromptBulk.py"]:::internal
        end
    end

    %% External Services
    subgraph "External Services"
        LLM["LLM Services"]:::external
        HIL["HIL System"]:::external
    end

    %% Output & Reporting Layer
    subgraph "Output & Reporting"
        direction TB
        RES["Result Store (results/)"]:::data
        subgraph "Tables"
            T1["actuator_stats.tex"]:::data
            T2["batch_2.0_stats.tex"]:::data
            T3["batch_3.0_stats.tex"]:::data
            T4["batch_5.0_stats.tex"]:::data
            T5["dropped_stats.tex"]:::data
            T6["single_stats.tex"]:::data
        end
        FIG["figures/"]:::data
    end

    %% Data Flows
    D -->|"FSRs.xlsx"| EXP1
    D --> EXP2
    D --> EXP3
    D --> EXP4
    D --> EXP5
    D --> EXP6
    D --> EXP7
    D --> EXP8
    D --> EXP9
    D --> EXP10
    D --> EXP11

    ENV -->|Env vars| EXP1
    TOML -->|Dependencies| EXP1
    LOCK --> EXP1
    REQ --> EXP1
    ENV --> EXP2
    TOML --> EXP2
    LOCK --> EXP2
    REQ --> EXP2
    ENV --> EXP3
    TOML --> EXP3
    LOCK --> EXP3
    REQ --> EXP3
    ENV --> EXP4
    TOML --> EXP4
    LOCK --> EXP4
    REQ --> EXP4
    ENV --> EXP5
    TOML --> EXP5
    LOCK --> EXP5
    REQ --> EXP5
    ENV --> EXP6
    TOML --> EXP6
    LOCK --> EXP6
    REQ --> EXP6
    ENV --> EXP7
    TOML --> EXP7
    LOCK --> EXP7
    REQ --> EXP7
    ENV --> EXP8
    TOML --> EXP8
    LOCK --> EXP8
    REQ --> EXP8
    ENV --> EXP9
    TOML --> EXP9
    LOCK --> EXP9
    REQ --> EXP9
    ENV --> EXP10
    TOML --> EXP10
    LOCK --> EXP10
    REQ --> EXP10
    ENV --> EXP11
    TOML --> EXP11
    LOCK --> EXP11
    REQ --> EXP11

    %% Orchestrator to TestGen
    EXP1 -->|"calls TestGen"| TGInit
    EXP2 --> TGInit
    EXP3 --> TGInit
    EXP4 --> TGInit
    EXP5 --> TGInit
    EXP6 --> TGInit
    EXP7 --> TGInit
    EXP8 --> TGInit
    EXP9 --> TGInit
    EXP10 --> TGInit
    EXP11 --> TGInit

    %% TestGen to LLM and back
    TGInit -->|"Prompt"| LLM
    AU --> LLM
    U --> LLM
    P1 --> LLM
    P2 --> LLM
    P3 --> LLM
    P4 --> LLM
    P5 --> LLM
    LLM -->|"Fault TCs"| TGInit

    %% Orchestrator executes on HIL
    TGInit -->|"Generated TCs"| EXP1
    LLM --> EXP1
    EXP1 -->|"Exec TCs"| HIL
    EXP2 -->|"Exec TCs"| HIL
    EXP3 -->|"Exec TCs"| HIL
    EXP4 -->|"Exec TCs"| HIL
    EXP5 -->|"Exec TCs"| HIL
    EXP6 -->|"Exec TCs"| HIL
    EXP7 -->|"Exec TCs"| HIL
    EXP8 -->|"Exec TCs"| HIL
    EXP9 -->|"Exec TCs"| HIL
    EXP10 -->|"Exec TCs"| HIL
    EXP11 -->|"Exec TCs"| HIL

    %% HIL to Results
    HIL -->|"Exec Logs"| RES

    %% Results to Reporting
    RES -->|"Load results"| T1
    RES --> T2
    RES --> T3
    RES --> T4
    RES --> T5
    RES --> T6
    RES --> FIG

    %% Click Events - Data & Config
    click D "https://github.com/ahmadhatahet/hil-fault-test-gen/tree/main/data/"
    click ENV "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/.env.template"
    click TOML "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/pyproject.toml"
    click LOCK "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/uv.lock"
    click REQ "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/requirements.in"

    %% Click Events - TestGen Library
    click TGInit "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/testgen/__init__.py"
    click AU "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/testgen/analysis_utils.py"
    click U "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/testgen/utils.py"
    click P1 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/testgen/prompts/Sensors.py"
    click P2 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/testgen/prompts/SensorActuator.py"
    click P3 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/testgen/prompts/SystemPrompt.py"
    click P4 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/testgen/prompts/UserPrompt.py"
    click P5 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/testgen/prompts/UserPromptBulk.py"

    %% Click Events - Notebooks
    click EXP1 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/experiment_bulk.ipynb"
    click EXP2 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/experiment_bulk_ollama.ipynb"
    click EXP3 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/experiment_all_single.ipynb"
    click EXP4 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/experiment_all_single_ollama.ipynb"
    click EXP5 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/experiment_all_single_drop_sensor.ipynb"
    click EXP6 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/experiment_all_single_drop_sensor_ollama.ipynb"
    click EXP7 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/experiment_actuator_sensor.ipynb"
    click EXP8 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/experiment_actuator_sensor_ollama.ipynb"
    click EXP9 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/analyze_actuators.ipynb"
    click EXP10 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/chat_analysis.ipynb"
    click EXP11 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/results_analysis.ipynb"

    %% Click Events - Output & Reporting
    click RES "https://github.com/ahmadhatahet/hil-fault-test-gen/tree/main/results/"
    click T1 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/tables/actuator_stats.tex"
    click T2 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/tables/batch_2.0_stats.tex"
    click T3 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/tables/batch_3.0_stats.tex"
    click T4 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/tables/batch_5.0_stats.tex"
    click T5 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/tables/dropped_stats.tex"
    click T6 "https://github.com/ahmadhatahet/hil-fault-test-gen/blob/main/tables/single_stats.tex"
    click FIG "https://github.com/ahmadhatahet/hil-fault-test-gen/tree/main/figures/"

    %% Styles
    classDef data fill:#ffe699,stroke:#333,stroke-width:1px
    classDef config fill:#fff2cc,stroke:#333,stroke-width:1px
    classDef internal fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef external fill:#cfc,stroke:#333,stroke-width:1px
```
