# git-disl/bert4eth architecture

> Architecture diagram of the GitHub repository git-disl/bert4eth, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/git-disl/bert4eth
- Repository: https://github.com/git-disl/bert4eth

## Overview

##### Project type and purpose
- This is a research‐oriented machine‐learning pipeline (“BERT4ETH”) for pre-training and fine-tuning a BERT-style Transformer on Ethereum transaction sequences to detect fraud (phishing) and perform de-anonymization tasks.
- Key indicators: a “Data/” folder with raw CSVs, a “Model/” folder containing data-generation and training scripts, and a README describing pre-training, fine-tuning, and evaluation.

##### File structure
- Top‐level:
- Data/ – raw transaction datasets (ENS pairs, phishing account lists, Tornado cash history)
- Material/ – PDFs and slide decks illustrating model architecture and multi-hop graphs
- Model/ – Python modules for sequence generation (gen_seq.py), pre-training data generation (gen_pretrain_data.py), model definition (modeling.py), optimization & training runners (run_pretrain.py, run_phishing_detection.py, run_dean_ENS.py, etc.), and utility scripts (vocab.py, output_embed.py)
- run.sh – orchestration entry point
- Presence of a JSON config (bert_config.json) signals a parameterized TensorFlow model.
- No frontend; this is purely a data‐processing + ML model codebase.

##### README for additional insights
- Architecture is a single‐stream pipeline:
a) Download & unzip raw data into Data/
b) Generate tokenized transaction sequences (gen_seq.py)
c) Mask & prepare MLM pre-training data (gen_pretrain_data.py)
d) Pre-train BERT4ETH in TensorFlow (run_pretrain.py)
e) Extract output embeddings (output_embed.py)
f) Downstream tasks: phishing detection (RF & DNN), ENS de-anonymization, Tornado Cash de-anonymization, and fine-tuning for phishing (run_finetune_phisher.py)
- Technologies: Python ≥3.6, TensorFlow 2.x, NumPy; uses in-house negative sampling strategies, cascaded MLP classifiers.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Data Sources
    External["Google Drive Data Sources"]:::external
    click External "https://github.com/git-disl/bert4eth/tree/master/Data/"

    %% Data Ingestion & Storage
    DataStore["Raw Data Store\n(Data/)"]:::dataStore
    click DataStore "https://github.com/git-disl/bert4eth/tree/master/Data/"

    %% Utilities
    Vocab["Vocabulary Utility\n(vocab.py)"]:::compute
    click Vocab "https://github.com/git-disl/bert4eth/blob/master/Model/vocab.py"

    %% Preprocessing & Sequence Generation
    Preproc["Sequence Generator\n(gen_seq.py)"]:::compute
    click Preproc "https://github.com/git-disl/bert4eth/blob/master/Model/gen_seq.py"

    %% Pre-training Data Generation
    GenPreData["Pre-training Data Generator\n(gen_pretrain_data.py)"]:::compute
    click GenPreData "https://github.com/git-disl/bert4eth/blob/master/Model/gen_pretrain_data.py"

    %% Compute & Training Cluster
    subgraph "Compute & Training" 
        Config["Configuration File\n(bert_config.json)"]:::config
        click Config "https://github.com/git-disl/bert4eth/blob/master/Model/bert_config.json"

        TrainingDef["Model Definition\n(modeling.py, optimization.py)"]:::compute
        click TrainingDef "https://github.com/git-disl/bert4eth/blob/master/Model/modeling.py"
        click TrainingDef "https://github.com/git-disl/bert4eth/blob/master/Model/optimization.py"

        TrainRunner["Pre-training Runner\n(run_pretrain.py)"]:::compute
        click TrainRunner "https://github.com/git-disl/bert4eth/blob/master/Model/run_pretrain.py"
    end

    %% Model Storage
    ModelStorage["Model Checkpoints\n(checkpointDir)"]:::storage

    %% Embedding Extraction
    EmbedExtractor["Embedding Extractor\n(output_embed.py)"]:::compute
    click EmbedExtractor "https://github.com/git-disl/bert4eth/blob/master/Model/output_embed.py"

    Embeddings["Embeddings Store"]:::dataStore

    %% Downstream Task Modules
    PhishRF["Phishing Detection RF\n(run_phishing_detection.py)"]:::compute
    click PhishRF "https://github.com/git-disl/bert4eth/blob/master/Model/run_phishing_detection.py"

    PhishDNN["Phishing Detection DNN\n(run_phishing_detection_dnn.py)"]:::compute
    click PhishDNN "https://github.com/git-disl/bert4eth/blob/master/Model/run_phishing_detection_dnn.py"

    ENSModule["ENS De-anonymization\n(run_dean_ENS.py)"]:::compute
    click ENSModule "https://github.com/git-disl/bert4eth/blob/master/Model/run_dean_ENS.py"

    TornadoModule["Tornado De-anonymization\n(run_dean_Tornado.py)"]:::compute
    click TornadoModule "https://github.com/git-disl/bert4eth/blob/master/Model/run_dean_Tornado.py"

    GenFinetuneData["Fine-tune Data Generator\n(gen_finetune_phisher_data.py)"]:::compute
    click GenFinetuneData "https://github.com/git-disl/bert4eth/blob/master/Model/gen_finetune_phisher_data.py"

    FinetuneRunner["Fine-tuning Runner\n(run_finetune_phisher.py)"]:::compute
    click FinetuneRunner "https://github.com/git-disl/bert4eth/blob/master/Model/run_finetune_phisher.py"

    %% Orchestration & Documentation
    Orchestration["Orchestration Script\n(run.sh)"]:::compute
    click Orchestration "https://github.com/git-disl/bert4eth/blob/master/run.sh"

    README["Project README\n(README.md)"]:::external
    click README "https://github.com/git-disl/bert4eth/blob/master/README.md"

    %% Data Flow
    External --> DataStore
    DataStore --> Vocab --> Preproc --> GenPreData --> TrainRunner
    Config --> TrainRunner
    TrainingDef --> TrainRunner
    TrainRunner --> ModelStorage
    ModelStorage --> EmbedExtractor --> Embeddings

    %% Downstream Flow
    Embeddings --> PhishRF
    Embeddings --> PhishDNN
    Embeddings --> ENSModule
    Embeddings --> TornadoModule
    ModelStorage --> FinetuneRunner
    Embeddings --> FinetuneRunner
    GenFinetuneData --> FinetuneRunner

    %% Orchestration triggers all stages
    Orchestration --> Preproc
    Orchestration --> GenPreData
    Orchestration --> TrainRunner
    Orchestration --> EmbedExtractor
    Orchestration --> PhishRF
    Orchestration --> PhishDNN
    Orchestration --> ENSModule
    Orchestration --> TornadoModule
    Orchestration --> GenFinetuneData
    Orchestration --> FinetuneRunner

    %% Styles
    classDef dataStore fill:#b2f2bb,stroke:#2f8f4e,color:#1b4332
    classDef compute fill:#a5d8ff,stroke:#339af0,color:#1c7ed6
    classDef storage fill:#ffd8a8,stroke:#fa8c16,color:#d9480f
    classDef external fill:#e9ecef,stroke:#868e96,color:#495057
    classDef config fill:#ffe066,stroke:#fab005,color:#b7791f
```
