# null-s-class/chainguard architecture

> Architecture diagram of the GitHub repository null-s-class/chainguard, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/null-s-class/chainguard
- Repository: https://github.com/null-s-class/chainguard

## Overview

##### Project Type and Purpose
- This is a machine learning project that fine-tunes the GraphCodeBERT model for code understanding tasks. It focuses on processing source code and bytecode data using custom preprocessing scripts, then training and evaluating the model.
- The README commands clearly indicate that the project leverages GraphCodeBERT (a transformer-based model for source code) with various configurations provided via CLI arguments. The project also appears to have multiple data processing workflows (e.g., cleaning CSV, JSONL creation, opcode processing) used to prepare data for training.

##### Key Components from the File Structure
- Data Processing:
- Directories “Processdata” and “data_processing” contain various scripts (e.g., Cleancsv.py, Createjsonl.py, opcode_proc.py, bytecode_proc.py, and source_code_proc.py) that are used to clean, preprocess, and transform the raw data. There are also scripts for dealing with bytecode and reentrancy labels, indicating some nuanced data preparation for specific code analysis tasks.
- Model & Training:
- Files like “model.py”, “run.py”, “run_edit_by_Null.py”, and “model_edit_by_Null.py” comprise the main entry points for training, inference, and testing. They orchestrate the use of the GraphCodeBERT model, and the README CLI snippets show how different operating modes (train, eval, test) are initiated.
- Utilities & Parsing:
- “Tree-sitter.py” is likely used for parsing source code using the Tree-sitter library, which suggests that the system integrates code syntax analysis.
- “utils.py” provides additional shared functionality.
- Configuration & Deployment:
- “requirements.txt” lists dependencies.
- “Dockerfile” and “docker-compose.yml” indicate containerization/deployment support.
- “dynamic_settings.txt” might be used for runtime configuration adjustments.
- Data Archives:
- Data archives like “Data.7z”, “Processdata.7z”, and “parser.7z” hint at datasets or preprocessed information that feed into the pipeline.

(a) Data Ingestion and Preprocessing:
- Component: "Raw Data Input & Archives"
- Represents the zipped datasets (Data.7z, Processdata.7z, etc.).
- Component: "Data Preprocessing Module"
- Sub-components: "Processdata Scripts" and "data_processing Scripts"
- Responsibilities: Clean CSV files, create JSONL, extract bytecode, process opcode and source code, remove comments/spaces, derive reentrancy labels, etc.
- Technologies: Python scripts, Tree-sitter for parsing.

(b) Model Training and Evaluation:
- Component: "Training Pipeline"
- Main scripts: run.py and variants.
- Model wrapper: "GraphCodeBERT" (configured via microsoft/graphcodebert-base parameters).
- Hyperparameter & Batch Size configurations (as per CLI arguments).
- Component: "Model Module"
- Files: model.py, model_edit_by_Null.py.
- Describes the model architecture, possibly extensions on GraphCodeBERT.

(c) Utilities and Core Services:
- Component: "Utility Services"
- Scripts: utils.py.
- Parsing functionality from "Tree-sitter.py" for code structure analysis.

(d) Deployment & Configuration:
- Component: "Containerization & Infrastructure"
- Files: Dockerfile, docker-compose.yml.
- Configuration files: dynamic_settings.txt, requirements.txt.

##### Relationships & Data Flow
- Results from preprocessing then feed into the training pipeline.
- The training pipeline uses model modules to train/evaluate the model.
- A control/configuration path flows from configuration files into both the preprocessing and model training components.
- Utility and parsing services support both preprocessing and model components.
- Deployment packaging (via Docker) wraps the entire pipeline to ensure consistency across environments.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion Layer
    subgraph "Data Ingestion"
        D_arch["Raw Data Archives (Data.7z, Processdata.7z, parser.7z)"]:::ingestion
    end

    %% Data Preprocessing Layer
    subgraph "Data Preprocessing"
        P1["Processdata Scripts"]:::preprocessing
        P2["Data Processing Scripts"]:::preprocessing
    end

    %% Model Training & Inference Layer
    subgraph "Model Training & Inference"
        T1["Model Training Pipeline (run.py, run_edit_by_Null.py)"]:::training
        T2["Model Module (model.py, model_edit_by_Null.py)"]:::training
    end

    %% Utility and Parsing Services
    subgraph "Utilities & Parsing Services"
        U1["Code Parsing Utility (Tree-sitter.py)"]:::utility
        U2["Miscellaneous Utility Services (utils.py)"]:::utility
    end

    %% Deployment and Configuration
    subgraph "Deployment & Configuration"
        C1["Deployment & Configuration (Dockerfile, docker-compose.yml, dynamic_settings.txt, requirements.txt)"]:::deployment
    end

    %% Data Flow Connections
    D_arch --> P1
    D_arch --> P2
    P1 --> T1
    P2 --> T1
    T1 --> T2

    %% Configuration Flows
    C1 --> P1
    C1 --> P2
    C1 --> T1

    %% Utility Support
    U1 --> P1
    U1 --> P2
    U2 --> P1
    U2 --> P2
    U2 --> T1

    %% Click Events for Component Mapping
    click D_arch "https://github.com/null-s-class/chainguard/blob/master/Data.7z"
    click D_arch "https://github.com/null-s-class/chainguard/blob/master/Processdata.7z"
    click D_arch "https://github.com/null-s-class/chainguard/blob/master/parser.7z"
    click P1 "https://github.com/null-s-class/chainguard/tree/master/Processdata/"
    click P2 "https://github.com/null-s-class/chainguard/tree/master/data_processing/"
    click T1 "https://github.com/null-s-class/chainguard/blob/master/run.py"
    click T1 "https://github.com/null-s-class/chainguard/blob/master/run_edit_by_Null.py"
    click T2 "https://github.com/null-s-class/chainguard/blob/master/model.py"
    click T2 "https://github.com/null-s-class/chainguard/blob/master/model_edit_by_Null.py"
    click U1 "https://github.com/null-s-class/chainguard/blob/master/Tree-sitter.py"
    click U2 "https://github.com/null-s-class/chainguard/blob/master/utils.py"
    click C1 "https://github.com/null-s-class/chainguard/tree/master/Dockerfile"
    click C1 "https://github.com/null-s-class/chainguard/blob/master/docker-compose.yml"
    click C1 "https://github.com/null-s-class/chainguard/blob/master/dynamic_settings.txt"
    click C1 "https://github.com/null-s-class/chainguard/blob/master/requirements.txt"

    %% Style Definitions
    classDef ingestion fill:#ffccff,stroke:#333,stroke-width:2px;
    classDef preprocessing fill:#ccffff,stroke:#333,stroke-width:2px;
    classDef training fill:#ccffcc,stroke:#333,stroke-width:2px;
    classDef utility fill:#ffcccc,stroke:#333,stroke-width:2px;
    classDef deployment fill:#ffffcc,stroke:#333,stroke-width:2px;
```
