# wahab4114/dibert architecture

> Architecture diagram of the GitHub repository wahab4114/dibert, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wahab4114/dibert
- Repository: https://github.com/wahab4114/dibert

## Overview

##### Project Type and Purpose
- This is an NLP research / training pipeline for “DIBERT,” a BERT-Base variant with an extra Parent Prediction (PP) objective.
- It comprises two major phases:
- Pretraining (on WikiText-103 with MLM, NSP, PP objectives)
- Downstream Finetuning & Evaluation (on tasks like QNLI, MRPC, SST-2, IMDB, SciTail, LIAR)

##### Directory Structure & Architectural Patterns
A. Top-level components
- config/           ← JSON configs for pretrain/finetune runs
- dibert.yml        ← environment/dependency spec
- datapreparation.py, preprocessing.py
- dataset.py        ← dataset abstraction for WikiText-103
- model.py          ← DIBERT model architecture (MLM + NSP + PP heads)
- train.py          ← pretraining orchestrator
- test_for_paper.py ← aggregated experiment runner / evaluation script
- utils.py          ← shared helper functions
- results/          ← pretraining result artifacts (models, logs)
- Downstreamtask/   ← one subfolder per finetuning task
– Each subfolder has:
· <task>config.py
· <task>dataset.py
· <task>model.py
· train_<task>.py
· test_<task>.py
· results/       ← finetuning outcome (params, logs, seeds)

B. Architectural patterns
- Modular pipeline: clean separation of data prep, model definition, training loops, and evaluation.
- Configuration-driven orchestration: JSON/YAML configs injected into scripts.
- Experiment tracking via filesystem hierarchy (results/…).

##### Key Insights from README
- model.py: core DIBERT network (BERT + PP head)
- datapreparation.py + dataset.py: building MLM, NSP, PP labels
- Downstreamtask/**: mirrored pipeline per task, finetuned on both BERT & DIBERT
- dibert.yml: reproducible environment
- utils.py: shared across both pretraining and finetuning

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Configuration & Utilities
    subgraph "Configuration & Utilities"
        direction TB
        B1["config/bert_base.json"]:::config
        B2["config/pretrain.json"]:::config
        B3["dibert.yml"]:::config
        U["utils.py"]:::config
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        Wiki["WikiText-103 (S3)"]:::data
    end

    %% Preprocessing & Dataset
    subgraph "Preprocessing & Dataset Abstraction"
        direction TB
        DP["datapreparation.py"]:::data
        PP["preprocessing.py"]:::data
        DS["dataset.py"]:::data
    end

    %% Pretraining Pipeline
    subgraph "Pretraining Pipeline"
        direction TB
        PT["train.py"]:::train
        M["model.py"]:::model
        RPT["results/model/"]:::train
    end

    %% Downstream Tasks
    subgraph "Downstream Tasks"
        direction TB

        subgraph "QNLI"
            direction TB
            QCfg["qnliconfig.py"]:::config
            QDS["qnlidataset.py"]:::data
            QM["qnlimodel.py"]:::model
            QTrain["train_qnli.py"]:::train
            QTest["test_qnli.py"]:::eval
            QRes["results/"]:::train
        end

        subgraph "MRPC"
            direction TB
            MRCfg["mprcconfig.py"]:::config
            MRDS["mrpcdataset.py"]:::data
            MRM["mrpcmodel.py"]:::model
            MRTrain["train_mrpc.py"]:::train
            MRTest["test_mrpc.py"]:::eval
            MRRes["results/"]:::train
        end

        subgraph "LIAR"
            direction TB
            LCfg["liarconfig.py"]:::config
            LDS["liardataset.py"]:::data
            LM["liarmodel.py"]:::model
            LTrain["train_liar.py"]:::train
            LTest["test_liar.py"]:::eval
            LRes["results/"]:::train
        end

        subgraph "SciTail"
            direction TB
            SCfg["scitailconfig.py"]:::config
            SDS["scitaildataset.py"]:::data
            SM["scitailmodel.py"]:::model
            STrain["train_scitail.py"]:::train
            STest["test_scitail.py"]:::eval
            SRes["results/"]:::train
        end

        subgraph "IMDB"
            direction TB
            ICfg["configimdb.py"]:::config
            IDS["imdbdataset.py"]:::data
            IM["imdbmodel.py"]:::model
            ITrain["train_imdb.py"]:::train
            ITest["testing_imdb.py"]:::eval
            IRes["results/"]:::train
        end

        subgraph "SST-2"
            direction TB
            SCfg2["configsst.py"]:::config
            SDS2["dataset.py"]:::data
            SM2["sstmodel.py"]:::model
            STTrain["train_sst.py"]:::train
            STTest["testing.py"]:::eval
            STRes["results/"]:::train
        end
    end

    %% Evaluation
    subgraph "Evaluation"
        direction TB
        EvalAll["test_for_paper.py"]:::eval
    end

    %% Data Flow & Dependencies
    Wiki --> DP --> PP --> DS
    DS --> PT --> M --> RPT
    PT --> U
    B1 --> PT
    B2 --> PT
    B3 --> PT
    U --> PT
    RPT --> EvalAll

    %% Downstream flow
    M --> QTrain --> QRes
    QCfg --> QTrain
    QDS --> QTrain
    QM --> QTrain
    QRes --> EvalAll

    M --> MRTrain --> MRRes
    MRCfg --> MRTrain
    MRDS --> MRTrain
    MRM --> MRTrain
    MRRes --> EvalAll

    M --> LTrain --> LRes
    LCfg --> LTrain
    LDS --> LTrain
    LM --> LTrain
    LRes --> EvalAll

    M --> STrain --> SRes
    SCfg --> STrain
    SDS --> STrain
    SM --> STrain
    SRes --> EvalAll

    M --> ITrain --> IRes
    ICfg --> ITrain
    IDS --> ITrain
    IM --> ITrain
    IRes --> EvalAll

    M --> STTrain --> STRes
    SCfg2 --> STTrain
    SDS2 --> STTrain
    SM2 --> STTrain
    STRes --> EvalAll

    EvalAll -->|"aggregate metrics"| EvalAll

    %% Click Events
    click B1 "https://github.com/wahab4114/dibert/blob/master/config/bert_base.json"
    click B2 "https://github.com/wahab4114/dibert/blob/master/config/pretrain.json"
    click B3 "https://github.com/wahab4114/dibert/blob/master/dibert.yml"
    click U "https://github.com/wahab4114/dibert/blob/master/utils.py"
    click DP "https://github.com/wahab4114/dibert/blob/master/datapreparation.py"
    click PP "https://github.com/wahab4114/dibert/blob/master/preprocessing.py"
    click DS "https://github.com/wahab4114/dibert/blob/master/dataset.py"
    click M "https://github.com/wahab4114/dibert/blob/master/model.py"
    click PT "https://github.com/wahab4114/dibert/blob/master/train.py"
    click EvalAll "https://github.com/wahab4114/dibert/blob/master/test_for_paper.py"
    click RPT "https://github.com/wahab4114/dibert/tree/master/results/model/"
    click QCfg "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/QNLI/qnliconfig.py"
    click QDS "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/QNLI/qnlidataset.py"
    click QM "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/QNLI/qnlimodel.py"
    click QTrain "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/QNLI/train_qnli.py"
    click QTest "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/QNLI/test_qnli.py"
    click QRes "https://github.com/wahab4114/dibert/tree/master/Downstreamtask/QNLI/results/"
    click MRCfg "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/MRPC/mprcconfig.py"
    click MRDS "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/MRPC/mrpcdataset.py"
    click MRM "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/MRPC/mrpcmodel.py"
    click MRTrain "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/MRPC/train_mrpc.py"
    click MRTest "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/MRPC/test_mrpc.py"
    click MRRes "https://github.com/wahab4114/dibert/tree/master/Downstreamtask/MRPC/results/"
    click LCfg "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/LIAR/liarconfig.py"
    click LDS "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/LIAR/liardataset.py"
    click LM "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/LIAR/liarmodel.py"
    click LTrain "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/LIAR/train_liar.py"
    click LTest "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/LIAR/test_liar.py"
    click LRes "https://github.com/wahab4114/dibert/tree/master/Downstreamtask/LIAR/results/"
    click SCfg "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/SciTail/scitailconfig.py"
    click SDS "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/SciTail/scitaildataset.py"
    click SM "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/SciTail/scitailmodel.py"
    click STrain "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/SciTail/train_scitail.py"
    click STest "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/SciTail/test_scitail.py"
    click SRes "https://github.com/wahab4114/dibert/tree/master/Downstreamtask/SciTail/results/"
    click ICfg "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/imdb/configimdb.py"
    click IDS "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/imdb/imdbdataset.py"
    click IM "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/imdb/imdbmodel.py"
    click ITrain "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/imdb/train_imdb.py"
    click ITest "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/imdb/testing_imdb.py"
    click IRes "https://github.com/wahab4114/dibert/tree/master/Downstreamtask/imdb/results/"
    click SCfg2 "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/sst2/configsst.py"
    click SDS2 "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/sst2/dataset.py"
    click SM2 "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/sst2/sstmodel.py"
    click STTrain "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/sst2/train_sst.py"
    click STTest "https://github.com/wahab4114/dibert/blob/master/Downstreamtask/sst2/testing.py"
    click STRes "https://github.com/wahab4114/dibert/tree/master/Downstreamtask/sst2/results/"

    %% Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef model fill:#ccffcc,stroke:#333,stroke-width:1px
    classDef train fill:#ffe5cc,stroke:#333,stroke-width:1px
    classDef eval fill:#e5ccff,stroke:#333,stroke-width:1px
    classDef config fill:#ffffcc,stroke:#333,stroke-width:1px
```
