# mlwithme/bertwithpretrained architecture

> Architecture diagram of the GitHub repository mlwithme/bertwithpretrained, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mlwithme/bertwithpretrained
- Repository: https://github.com/mlwithme/bertwithpretrained

## Overview

BertWithPretrained is a PyTorch BERT implementation and task-training suite. Users provide task datasets and pretrained model files, run a task script, and receive trained or inferred classification, entailment, multiple-choice, question-answering, NER, or pretraining results. Task-specific dataset loaders prepare examples; task heads use the shared BERT encoder. The graph preserves the README’s documented capabilities, but several task implementations and loader-to-task calls were not excerpted, so those relationships are documented at the README level rather than inferred from unsampled code.

Main entry points: [`model/BasicBert/Bert.py`](https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/Bert.py) (BERT encoder), [`utils/data_helpers.py`](https://github.com/mlwithme/bertwithpretrained/blob/main/utils/data_helpers.py) (Task dataset loaders), [`bert_base_chinese`](https://github.com/mlwithme/bertwithpretrained/tree/main/bert_base_chinese) (Pretrained files), [`Tasks/TaskForSingleSentenceClassification.py`](https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForSingleSentenceClassification.py) (Chinese classification)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Task workflows"]
  node_task_single["Chinese classification"]
  node_task_pair["MNLI entailment"]
  node_task_choice["SWAG choices"]
  node_task_qa["SQuAD answers"]
  node_task_ner["Chinese NER"]
  node_task_pretrain["BERT pretraining"]
end

subgraph group_data["Data preparation"]
  node_datasets["Task datasets"]
  node_pretrained["Pretrained files"]
  node_task_loaders["Task dataset loaders<br/>[data_helpers.py]"]
  node_pretrain_data["Pretraining data builder"]
end

subgraph group_heads["Task heads"]
  node_sentence_head["Sentence classifier"]
  node_choice_head["Multiple-choice head"]
  node_qa_head["Question-answer head"]
  node_token_head["Token classifier"]
  node_pretrain_head["NSP and MLM heads"]
end

subgraph group_bert["BERT core"]
  node_bert_model["BERT encoder<br/>[Bert.py]"]
  node_bert_embedding["Input embeddings<br/>[BertEmbedding.py]"]
  node_transformer["Attention layers<br/>[MyTransformer.py]"]
  node_config["BERT configuration<br/>[BertConfig.py]"]
end

node_user(("Engineer"))

node_user -->|"runs"| node_task_single
node_user -->|"runs"| node_task_pair
node_user -->|"runs"| node_task_choice
node_user -->|"runs"| node_task_qa
node_user -->|"runs"| node_task_ner
node_user -->|"runs"| node_task_pretrain
node_datasets -->|"provides examples"| node_task_loaders
node_task_loaders -->|"prepares input"| node_task_single
node_task_loaders -->|"prepares input"| node_task_pair
node_task_loaders -->|"prepares input"| node_task_choice
node_task_loaders -->|"prepares input"| node_task_qa
node_task_loaders -->|"prepares input"| node_task_ner
node_pretrain_data -->|"prepares examples"| node_task_pretrain
node_task_single -->|"trains or infers"| node_sentence_head
node_task_pair -->|"trains or infers"| node_sentence_head
node_task_choice -->|"trains or infers"| node_choice_head
node_task_qa -->|"trains or infers"| node_qa_head
node_task_ner -->|"trains or infers"| node_token_head
node_task_pretrain -->|"trains"| node_pretrain_head
node_sentence_head -->|"calls encoder"| node_bert_model
node_choice_head -->|"calls encoder"| node_bert_model
node_qa_head -->|"calls encoder"| node_bert_model
node_token_head -->|"calls encoder"| node_bert_model
node_pretrain_head -->|"uses encoder"| node_bert_model
node_bert_model -->|"uses embeddings"| node_bert_embedding
node_bert_model -->|"uses attention"| node_transformer
node_bert_model -.->|"configured by"| node_config
node_task_single -.->|"loads weights"| node_pretrained
node_task_pair -.->|"loads weights"| node_pretrained
node_task_choice -.->|"loads weights"| node_pretrained
node_task_qa -.->|"loads weights"| node_pretrained
node_task_ner -.->|"loads weights"| node_pretrained

click node_datasets "https://github.com/mlwithme/bertwithpretrained/tree/main/data"
click node_pretrained "https://github.com/mlwithme/bertwithpretrained/tree/main/bert_base_chinese"
click node_task_single "https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForSingleSentenceClassification.py"
click node_task_pair "https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForPairSentenceClassification.py"
click node_task_choice "https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForMultipleChoice.py"
click node_task_qa "https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForSQuADQuestionAnswering.py"
click node_task_ner "https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForChineseNER.py"
click node_task_pretrain "https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForPretraining.py"
click node_task_loaders "https://github.com/mlwithme/bertwithpretrained/blob/main/utils/data_helpers.py"
click node_pretrain_data "https://github.com/mlwithme/bertwithpretrained/blob/main/utils/create_pretraining_data.py"
click node_sentence_head "https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForSentenceClassification.py"
click node_choice_head "https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForMultipleChoice.py"
click node_qa_head "https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForQuestionAnswering.py"
click node_token_head "https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForTokenClassification.py"
click node_pretrain_head "https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForNSPAndMLM.py"
click node_bert_model "https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/Bert.py"
click node_bert_embedding "https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/BertEmbedding.py"
click node_transformer "https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/MyTransformer.py"
click node_config "https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/BertConfig.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_task_single,node_task_pair,node_task_choice,node_task_qa,node_task_ner,node_task_pretrain toneBlue
class node_datasets,node_pretrained,node_task_loaders,node_pretrain_data toneAmber
class node_sentence_head,node_choice_head,node_qa_head,node_token_head,node_pretrain_head toneMint
class node_bert_model,node_bert_embedding,node_transformer,node_config toneRose
class node_user toneIndigo
```

## Components

### Task workflows

- [Chinese classification](https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForSingleSentenceClassification.py): `Tasks/TaskForSingleSentenceClassification.py`
- [MNLI entailment](https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForPairSentenceClassification.py): `Tasks/TaskForPairSentenceClassification.py`
- [SWAG choices](https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForMultipleChoice.py): `Tasks/TaskForMultipleChoice.py`
- [SQuAD answers](https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForSQuADQuestionAnswering.py): `Tasks/TaskForSQuADQuestionAnswering.py`
- [Chinese NER](https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForChineseNER.py): `Tasks/TaskForChineseNER.py`
- [BERT pretraining](https://github.com/mlwithme/bertwithpretrained/blob/main/Tasks/TaskForPretraining.py): `Tasks/TaskForPretraining.py`

### Data preparation

- [Task datasets](https://github.com/mlwithme/bertwithpretrained/tree/main/data): `data`
- [Pretrained files](https://github.com/mlwithme/bertwithpretrained/tree/main/bert_base_chinese): `bert_base_chinese`
- [Task dataset loaders](https://github.com/mlwithme/bertwithpretrained/blob/main/utils/data_helpers.py): `utils/data_helpers.py`
- [Pretraining data builder](https://github.com/mlwithme/bertwithpretrained/blob/main/utils/create_pretraining_data.py): `utils/create_pretraining_data.py`

### Task heads

- [Sentence classifier](https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForSentenceClassification.py): `model/DownstreamTasks/BertForSentenceClassification.py`
- [Multiple-choice head](https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForMultipleChoice.py): `model/DownstreamTasks/BertForMultipleChoice.py`
- [Question-answer head](https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForQuestionAnswering.py): `model/DownstreamTasks/BertForQuestionAnswering.py`
- [Token classifier](https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForTokenClassification.py): `model/DownstreamTasks/BertForTokenClassification.py`
- [NSP and MLM heads](https://github.com/mlwithme/bertwithpretrained/blob/main/model/DownstreamTasks/BertForNSPAndMLM.py): `model/DownstreamTasks/BertForNSPAndMLM.py`

### BERT core

- [BERT encoder](https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/Bert.py): `model/BasicBert/Bert.py`
- [Input embeddings](https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/BertEmbedding.py): `model/BasicBert/BertEmbedding.py`
- [Attention layers](https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/MyTransformer.py): `model/BasicBert/MyTransformer.py`
- [BERT configuration](https://github.com/mlwithme/bertwithpretrained/blob/main/model/BasicBert/BertConfig.py): `model/BasicBert/BertConfig.py`

### Other components

- **Engineer**

## Connections

- Engineer → Chinese classification: runs (evidence: `README.md`)
- Engineer → MNLI entailment: runs (evidence: `README.md`)
- Engineer → SWAG choices: runs (evidence: `README.md`)
- Engineer → SQuAD answers: runs (evidence: `README.md`)
- Engineer → Chinese NER: runs (evidence: `README.md`)
- Engineer → BERT pretraining: runs (evidence: `README.md`)
- Task datasets → Task dataset loaders: provides examples (evidence: `README.md`)
- Task dataset loaders → Chinese classification: prepares input (evidence: `README.md`)
- Task dataset loaders → MNLI entailment: prepares input (evidence: `README.md`)
- Task dataset loaders → SWAG choices: prepares input (evidence: `README.md`)
- Task dataset loaders → SQuAD answers: prepares input (evidence: `README.md`)
- Task dataset loaders → Chinese NER: prepares input (evidence: `README.md`)
- Pretraining data builder → BERT pretraining: prepares examples (evidence: `README.md`)
- Chinese classification → Sentence classifier: trains or infers (evidence: `README.md`)
- MNLI entailment → Sentence classifier: trains or infers (evidence: `README.md`)
- SWAG choices → Multiple-choice head: trains or infers (evidence: `README.md`)
- SQuAD answers → Question-answer head: trains or infers (evidence: `README.md`)
- Chinese NER → Token classifier: trains or infers (evidence: `README.md`)
- BERT pretraining → NSP and MLM heads: trains (evidence: `README.md`)
- Sentence classifier → BERT encoder: calls encoder (evidence: `model/DownstreamTasks/BertForSentenceClassification.py`)
- Multiple-choice head → BERT encoder: calls encoder (evidence: `model/DownstreamTasks/BertForMultipleChoice.py`)
- Question-answer head → BERT encoder: calls encoder (evidence: `model/DownstreamTasks/BertForQuestionAnswering.py`)
- Token classifier → BERT encoder: calls encoder (evidence: `model/DownstreamTasks/BertForTokenClassification.py`)
- NSP and MLM heads → BERT encoder: uses encoder (evidence: `model/DownstreamTasks/BertForNSPAndMLM.py`)
- BERT encoder → Input embeddings: uses embeddings (evidence: `model/BasicBert/Bert.py`)
- BERT encoder → Attention layers: uses attention (evidence: `model/BasicBert/Bert.py`)
- BERT encoder → BERT configuration: configured by
- Chinese classification → Pretrained files: loads weights
- MNLI entailment → Pretrained files: loads weights
- SWAG choices → Pretrained files: loads weights
- SQuAD answers → Pretrained files: loads weights
- Chinese NER → Pretrained files: loads weights