# feisun/bert4rec architecture

> Architecture diagram of the GitHub repository feisun/bert4rec, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/feisun/bert4rec
- Repository: https://github.com/feisun/bert4rec

## Overview

##### Main Components
- "Data Layer" – This includes the source files under the data directory (e.g., beauty.txt, ml-1m.txt, ml-20m.zip, steam.txt). These represent the raw user-item interactions or click logs.
- "Data Generation/Preprocessing Module" – Represented by gen_data.py and gen_data_fin.py. This module takes raw text data and generates the masked training input (TFRecords, versioned vocab files, and user history files) as seen in the README command examples.
- "Configuration Module" – The collection of JSON files in the bert_train directory (e.g., bert_config_ml-1m_64.json, others) which set hyper-parameter values such as hidden size, dropout, number of layers, etc.
- "Modeling Module" – The core of the project, represented by modeling.py, which likely contains the transformer network definitions, and optimization.py, which handles loss computation and training optimizations.
- "Training Pipeline" – run.py is the main driver for training the model using TensorFlow. This module loads the generated data, applies the configuration from the JSON files, and executes both training and evaluation steps.
- "Utility Modules" – Include util.py and vocab.py, which are responsible for various helper functions such as logging, processing vocabulary, and other common tasks.
- "Shell Scripts" – run_beauty.sh, run_ml-1m.sh, etc., which orchestrate the commands needed to generate data and kick off model training. These serve as entry points customized per dataset.

##### Relationships and Data Flow
- That the "Data Generation Module" produces several artifacts (training tfrecord files, test tfrecord files, vocab files, and history files) which feed into the "Training Pipeline."
- The "Training Pipeline" (run.py) should be depicted as central, loading both training data (from the data generation output) and hyper-parameters (from the "Configuration Module").
- Within the "Training Pipeline," illustrate an internal flow where the "Modeling Module" (model definitions from modeling.py) and "Optimization Module" (from optimization.py) operate together to train the BERT-based model.
- The "Utility Modules" can be shown as supportive blocks connected to both the data generation and training components.
- Optionally, include a note that the shell scripts serve as the start-up points, triggering the entire process.

##### Architectural Patterns and Technologies
- The architecture follows a modular design – each component encapsulates a distinct function (data preprocessing, model definition, optimization, etc.). Emphasize loose coupling where configuration and data generation are separate from training.
- Tools/Technologies include Python (2.7+), TensorFlow 1.12 (with GPU support), CUDA, and JSON for configuration.
- That this pipeline adopts a pre-training data generation step (masking for BERT) followed by transformer-based training, an approach common in modern deep learning applications.

- Annotations where necessary (for instance, include configuration details like hyper-parameter settings coming from the JSON files).

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Shell Scripts Subgraph
    subgraph "Shell Scripts"
        s1["run_beauty.sh"]:::script
        s2["run_ml-1m.sh"]:::script
        s3["run_ml-20m.sh"]:::script
        s4["run_steam.sh"]:::script
    end

    %% Data Layer
    r["Raw Data (data)"]:::rawData

    %% Data Generation/Preprocessing Subgraph
    subgraph "Data Generation/Preprocessing Module"
        d1["gen_data.py"]:::module
        d2["gen_data_fin.py"]:::module
    end

    %% Utility Modules Subgraph
    subgraph "Utility Modules"
        u1["util.py"]:::module
        u2["vocab.py"]:::module
    end

    %% Configuration Module
    c["Configuration Module (bert_train/)"]:::config

    %% Training Pipeline and Internal Modules
    t["Training Pipeline (run.py)"]:::module
    m["Modeling Module (modeling.py)"]:::module
    o["Optimization Module (optimization.py)"]:::module

    %% Connections

    %% Shell Scripts trigger Data Generation and Training Pipeline
    s1 -->|"triggers"| d1
    s2 -->|"triggers"| d1
    s3 -->|"triggers"| d1
    s4 -->|"triggers"| d1

    s1 -->|"triggers"| t
    s2 -->|"triggers"| t
    s3 -->|"triggers"| t
    s4 -->|"triggers"| t

    %% Raw Data flows into Data Generation
    r -->|"raw data"| d1
    r -->|"raw data"| d2

    %% Data Generation outputs feed into Training Pipeline
    d1 -->|"generates training artifacts"| t
    d2 -->|"generates training artifacts"| t

    %% Utility Modules assist Data Generation and Training Pipeline
    u1 ---|"helper functions"| d1
    u2 ---|"helper functions"| d1
    u1 ---|"helper functions"| t
    u2 ---|"helper functions"| t

    %% Configuration feeds into Training Pipeline
    c -->|"provides hyper-parameters"| t

    %% Training Pipeline invokes Modeling and Optimization
    t -->|"trains model using"| m
    t -->|"optimizes training with"| o

    %% Styles
    classDef rawData fill:#ffeb3b,stroke:#333,stroke-width:2px;
    classDef module fill:#8bc34a,stroke:#333,stroke-width:2px;
    classDef script fill:#03a9f4,stroke:#333,stroke-width:2px;
    classDef config fill:#ff9800,stroke:#333,stroke-width:2px;

    %% Click Events
    click r "https://github.com/feisun/bert4rec/tree/master/data/"
    click d1 "https://github.com/feisun/bert4rec/blob/master/gen_data.py"
    click d2 "https://github.com/feisun/bert4rec/blob/master/gen_data_fin.py"
    click c "https://github.com/feisun/bert4rec/tree/master/bert_train/"
    click t "https://github.com/feisun/bert4rec/blob/master/run.py"
    click m "https://github.com/feisun/bert4rec/blob/master/modeling.py"
    click o "https://github.com/feisun/bert4rec/blob/master/optimization.py"
    click u1 "https://github.com/feisun/bert4rec/blob/master/util.py"
    click u2 "https://github.com/feisun/bert4rec/blob/master/vocab.py"
    click s1 "https://github.com/feisun/bert4rec/blob/master/run_beauty.sh"
    click s2 "https://github.com/feisun/bert4rec/blob/master/run_ml-1m.sh"
    click s3 "https://github.com/feisun/bert4rec/blob/master/run_ml-20m.sh"
    click s4 "https://github.com/feisun/bert4rec/blob/master/run_steam.sh"
```
