# samyamoyrakshit/papers-from-scratch architecture

> Architecture diagram of the GitHub repository samyamoyrakshit/papers-from-scratch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/samyamoyrakshit/papers-from-scratch
- Repository: https://github.com/samyamoyrakshit/papers-from-scratch

## Overview

This repository is a collection of four from-scratch PyTorch model replications, not one shared serving pipeline: BERT for Bengali news classification, GPT-2 for Bengali text generation, Transformer for English-to-Bengali translation, and ViT for temple-image classification. The README documents their data and task workflows; sampled source confirms selected model composition, BPE tokenization, and the BERT demo’s call into inference. Training, evaluation, and some ViT model internals were not sampled, so those links are documented rather than source-verified.

Main entry points: [`BERT/models/bert.py`](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/models/bert.py) (BERT encoder), [`GPT/models/gpt.py`](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/models/gpt.py) (GPT language model), [`GPT/bpe/bpe.py`](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/bpe/bpe.py) (Byte-level BPE), [`transformer/models/transformer.py`](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/models/transformer.py) (Seq2seq Transformer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_bert["BERT classification"]
  node_bertdata["BERT corpus and data<br/>[data_utils.py]"]
  node_bertpretrain["BERT pretraining<br/>[pretrain.py]"]
  node_bertmodel["BERT encoder<br/>[bert.py]"]
  node_bertfinetune["Classification fine-tuning<br/>[finetune.py]"]
  node_bertfinetunedata["Labeled data batching<br/>[finetune_data.py]"]
  node_bertapp["News classifier demo<br/>[app.py]"]
  node_bertinfer["Classification inference<br/>[inference.py]"]
end

subgraph group_gpt["GPT-2 generation"]
  node_bpe["Byte-level BPE<br/>[bpe.py]"]
  node_bytecode["Byte coding<br/>[byte_coder.py]"]
  node_pretokenize["Pre-tokenization<br/>[pretokenize.py]"]
  node_gptdata["GPT corpus and data<br/>[data_utils.py]"]
  node_gptpretrain["GPT pretraining<br/>[pretrain.py]"]
  node_gptmodel["GPT language model<br/>[gpt.py]"]
  node_gptgenerate["Text generation<br/>[generate.py]"]
end

subgraph group_mt["Transformer translation"]
  node_mtdata["Translation data<br/>[data_utils.py]"]
  node_mttrain["Model training<br/>[train.py]"]
  node_mtmodel["Seq2seq Transformer<br/>[transformer.py]"]
  node_mtinfer["Translation inference<br/>[inference.py]"]
end

subgraph group_vit["ViT images"]
  node_vitdata["Image data setup<br/>[data_setup.py]"]
  node_vitsplit["Dataset splitting<br/>[split_dataset.py]"]
  node_vitengine["Image training<br/>[engine.py]"]
  node_vitpredict["Image predictions<br/>[predictions.py]"]
end

subgraph group_shared["Shared model layers"]
  node_attention["Attention and layers"]
  node_layernorm["Layer normalization<br/>[layer_norm.py]"]
end

node_user(("User"))
node_wiki["Bengali Wikipedia"]
node_indicglue["IndicGLUE sna.bn"]
node_samanantar["Samanantar corpus"]
node_templedata["Temple images"]

node_wiki -->|"supplies corpus"| node_bertdata
node_bertdata -.->|"prepares data"| node_bertpretrain
node_bertpretrain -.->|"trains"| node_bertmodel
node_indicglue -->|"supplies labels"| node_bertfinetunedata
node_bertfinetunedata -->|"batches examples"| node_bertfinetune
node_bertfinetune -.->|"fine-tunes"| node_bertmodel
node_user -->|"submits text"| node_bertapp
node_bertapp -->|"calls inference"| node_bertinfer
node_bertmodel -.->|"reuses attention"| node_attention
node_wiki -->|"supplies corpus"| node_gptdata
node_gptdata -.->|"prepares data"| node_gptpretrain
node_gptpretrain -.->|"trains"| node_gptmodel
node_gptmodel -.->|"uses tokenizer"| node_bpe
node_bpe -->|"encodes bytes"| node_bytecode
node_bpe -->|"splits text"| node_pretokenize
node_user -.->|"requests generation"| node_gptgenerate
node_gptgenerate -.->|"generates with"| node_gptmodel
node_samanantar -->|"supplies pairs"| node_mtdata
node_mtdata -.->|"provides batches"| node_mttrain
node_mttrain -.->|"trains"| node_mtmodel
node_user -.->|"requests translation"| node_mtinfer
node_mtinfer -.->|"decodes with"| node_mtmodel
node_mtmodel -->|"uses layers"| node_attention
node_templedata -->|"supplies images"| node_vitdata
node_vitdata -.->|"prepares splits"| node_vitsplit
node_vitsplit -.->|"provides datasets"| node_vitengine
node_vitengine -.->|"produces predictions"| node_vitpredict
node_bertmodel -->|"uses normalization"| node_layernorm
node_gptmodel -->|"uses normalization"| node_layernorm

click node_bertdata "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/utils/data_utils.py"
click node_bertpretrain "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/scripts/pretrain.py"
click node_bertmodel "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/models/bert.py"
click node_bertfinetune "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/scripts/finetune.py"
click node_bertfinetunedata "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/utils/finetune_data.py"
click node_bertapp "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/scripts/app.py"
click node_bertinfer "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/scripts/inference.py"
click node_bpe "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/bpe/bpe.py"
click node_bytecode "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/bpe/byte_coder.py"
click node_pretokenize "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/bpe/pretokenize.py"
click node_gptdata "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/utils/data_utils.py"
click node_gptpretrain "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/scripts/pretrain.py"
click node_gptmodel "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/models/gpt.py"
click node_gptgenerate "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/scripts/generate.py"
click node_mtdata "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/utils/data_utils.py"
click node_mttrain "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/scripts/train.py"
click node_mtmodel "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/models/transformer.py"
click node_mtinfer "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/scripts/inference.py"
click node_vitdata "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/ViT/utils/data_setup.py"
click node_vitsplit "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/ViT/utils/split_dataset.py"
click node_vitengine "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/ViT/utils/engine.py"
click node_vitpredict "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/ViT/utils/predictions.py"
click node_attention "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/models/modules/multi_head_attention.py"
click node_layernorm "https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/models/modules/layer_norm.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_bertdata,node_bertpretrain,node_bertmodel,node_bertfinetune,node_bertfinetunedata,node_bertapp,node_bertinfer,node_user toneBlue
class node_bpe,node_bytecode,node_pretokenize,node_gptdata,node_gptpretrain,node_gptmodel,node_gptgenerate toneAmber
class node_mtdata,node_mttrain,node_mtmodel,node_mtinfer toneMint
class node_vitdata,node_vitsplit,node_vitengine,node_vitpredict toneRose
class node_attention,node_layernorm,node_wiki,node_indicglue,node_samanantar,node_templedata toneIndigo
```

## Components

### BERT classification

- [BERT corpus and data](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/utils/data_utils.py): `BERT/utils/data_utils.py`
- [BERT pretraining](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/scripts/pretrain.py): `BERT/scripts/pretrain.py`
- [BERT encoder](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/models/bert.py): `BERT/models/bert.py`
- [Classification fine-tuning](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/scripts/finetune.py): `BERT/scripts/finetune.py`
- [Labeled data batching](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/utils/finetune_data.py): `BERT/utils/finetune_data.py`
- [News classifier demo](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/scripts/app.py): `BERT/scripts/app.py`
- [Classification inference](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/BERT/scripts/inference.py): `BERT/scripts/inference.py`

### GPT-2 generation

- [Byte-level BPE](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/bpe/bpe.py): `GPT/bpe/bpe.py`
- [Byte coding](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/bpe/byte_coder.py): `GPT/bpe/byte_coder.py`
- [Pre-tokenization](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/bpe/pretokenize.py): `GPT/bpe/pretokenize.py`
- [GPT corpus and data](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/utils/data_utils.py): `GPT/utils/data_utils.py`
- [GPT pretraining](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/scripts/pretrain.py): `GPT/scripts/pretrain.py`
- [GPT language model](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/models/gpt.py): `GPT/models/gpt.py`
- [Text generation](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/GPT/scripts/generate.py): `GPT/scripts/generate.py`

### Transformer translation

- [Translation data](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/utils/data_utils.py): `transformer/utils/data_utils.py`
- [Model training](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/scripts/train.py): `transformer/scripts/train.py`
- [Seq2seq Transformer](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/models/transformer.py): `transformer/models/transformer.py`
- [Translation inference](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/scripts/inference.py): `transformer/scripts/inference.py`

### ViT images

- [Image data setup](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/ViT/utils/data_setup.py): `ViT/utils/data_setup.py`
- [Dataset splitting](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/ViT/utils/split_dataset.py): `ViT/utils/split_dataset.py`
- [Image training](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/ViT/utils/engine.py): `ViT/utils/engine.py`
- [Image predictions](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/ViT/utils/predictions.py): `ViT/utils/predictions.py`

### Shared model layers

- [Attention and layers](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/models/modules/multi_head_attention.py): `transformer/models/modules/multi_head_attention.py`
- [Layer normalization](https://github.com/samyamoyrakshit/papers-from-scratch/blob/main/transformer/models/modules/layer_norm.py): `transformer/models/modules/layer_norm.py`

### Other components

- **User**
- **Bengali Wikipedia**
- **IndicGLUE sna.bn**
- **Samanantar corpus**
- **Temple images**

## Connections

- Bengali Wikipedia → BERT corpus and data: supplies corpus (evidence: `README.md`)
- BERT corpus and data → BERT pretraining: prepares data
- BERT pretraining → BERT encoder: trains
- IndicGLUE sna.bn → Labeled data batching: supplies labels (evidence: `README.md`)
- Labeled data batching → Classification fine-tuning: batches examples (evidence: `BERT/utils/finetune_data.py`)
- Classification fine-tuning → BERT encoder: fine-tunes
- User → News classifier demo: submits text (evidence: `BERT/scripts/app.py`)
- News classifier demo → Classification inference: calls inference (evidence: `BERT/scripts/app.py`)
- BERT encoder → Attention and layers: reuses attention
- Bengali Wikipedia → GPT corpus and data: supplies corpus (evidence: `README.md`)
- GPT corpus and data → GPT pretraining: prepares data
- GPT pretraining → GPT language model: trains
- GPT language model → Byte-level BPE: uses tokenizer
- Byte-level BPE → Byte coding: encodes bytes (evidence: `GPT/bpe/bpe.py`)
- Byte-level BPE → Pre-tokenization: splits text (evidence: `GPT/bpe/bpe.py`)
- User → Text generation: requests generation
- Text generation → GPT language model: generates with
- Samanantar corpus → Translation data: supplies pairs (evidence: `README.md`)
- Translation data → Model training: provides batches
- Model training → Seq2seq Transformer: trains
- User → Translation inference: requests translation
- Translation inference → Seq2seq Transformer: decodes with
- Seq2seq Transformer → Attention and layers: uses layers (evidence: `transformer/models/transformer.py`)
- Temple images → Image data setup: supplies images (evidence: `README.md`)
- Image data setup → Dataset splitting: prepares splits
- Dataset splitting → Image training: provides datasets
- Image training → Image predictions: produces predictions
- BERT encoder → Layer normalization: uses normalization
- GPT language model → Layer normalization: uses normalization (evidence: `GPT/models/gpt.py`)