# ryderling/malgraph architecture

> Architecture diagram of the GitHub repository ryderling/malgraph, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ryderling/malgraph
- Repository: https://github.com/ryderling/malgraph

## Overview

##### Main Components

- Data Preprocessing Module
- Located mainly in the samples folder (PreProcess.py and sample files). This module interfaces with IDA Pro to parse disassembled executable files and converts the raw data (using the function parse_json_list_2_pyg_object()) into PyTorch Geometric data objects.

- Vocabulary Builder
- Found in the src/utils/Vocabulary.py file. This component is vital for constructing and saving the vocabulary of external function names across the training dataset.

- Configuration Management
- The configs/default.yaml file contains hyper-parameters and settings for both preprocessing and model training. This file drives how the training process is initialized.

- Model Architecture
- In the src/models folder, the HierarchicalGraphModel.py provides the core model implementation, capturing the hierarchical structure (inter-function call graph plus intra-function control flow graphs).

- Model Training & Execution
- The src/DistTrainModel.py script is the entry point for training the model. It utilizes the preprocessed data, configured parameters, and the defined model architecture.

- Auxiliary Utilities
- Under src/utils, additional helper modules (like FunctionHelpers.py, ParameterClasses.py, PreProcessedDataset.py, RealBatch.py) facilitate data manipulation, batching, and parameter management.

##### Relationships and Data Flow

- The Configuration Management component (configs/default.yaml) should be displayed adjacent to both the preprocessing and training modules, emphasizing that hyper-parameters are shared across the pipeline.

- Finally, the Model architecture (HierarchicalGraphModel) is central to processing the graph data, with training routines (DistTrainModel.py) orchestrating the training process.

##### Architectural Patterns and Technologies

- Clearly separate concerns between data ingestion (preprocessing), data transformation (vocabulary building and dataset preparations), and the core model training.
- That the system employs a hierarchical model design, where data is represented on multiple abstraction levels (inter-function and intra-function graphs).
- Use of PyTorch Geometric for graph data management and neural computations—a critical dependency in the design.
- The design implicitly follows a modular structure, making it easier to enhance or replace individual components (e.g., upgrading preprocessing or experimenting with different graph models).

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["IDA Pro"]:::external
    D["Configuration Management (configs/default.yaml)"]:::config
    B["Data Preprocessing (samples/PreProcess.py)"]:::module
    C["Vocabulary Builder (src/utils/Vocabulary.py)"]:::module
    E["HierarchicalGraphModel (src/models/HierarchicalGraphModel.py)"]:::module
    F["Model Training (src/DistTrainModel.py)"]:::training
    G["Auxiliary Utilities (src/utils)"]:::utils

    A -->|"parsesExe"| B
    D -->|"configures"| B
    B -->|"graphData"| C
    B -->|"graphData"| F
    D -->|"configures"| F
    C -->|"vocabProvides"| F
    E -->|"modelStructure"| F
    G -.->|"helpers"| B
    G -.->|"helpers"| F

    click B "https://github.com/ryderling/malgraph/blob/main/samples/PreProcess.py"
    click C "https://github.com/ryderling/malgraph/blob/main/src/utils/Vocabulary.py"
    click D "https://github.com/ryderling/malgraph/blob/main/configs/default.yaml"
    click E "https://github.com/ryderling/malgraph/blob/main/src/models/HierarchicalGraphModel.py"
    click F "https://github.com/ryderling/malgraph/blob/main/src/DistTrainModel.py"
    click G "https://github.com/ryderling/malgraph/tree/main/src/utils"

    classDef external fill:#FFEEAD,stroke:#333,stroke-width:2px
    classDef module fill:#AEE1F9,stroke:#333,stroke-width:2px
    classDef training fill:#D3F9B5,stroke:#333,stroke-width:2px
    classDef config fill:#FAD4BA,stroke:#333,stroke-width:2px
    classDef utils fill:#E2C2F9,stroke:#333,stroke-width:2px
```
