samyamoyrakshit/papers-from-scratch

How samyamoyrakshit/papers-from-scratch works

This repository is a collection of four from-scratch PyTorch model replications, not one shared serving pipeline: BERT for Bengali news classification, GPT-2 for Bengali text generation, Transformer for English-to-Bengali translation, and ViT for temple-image classification. The README documents their data and task workflows; sampled source confirms selected model composition, BPE tokenization, and the BERT demo’s call into inference. Training, evaluation, and some ViT model internals were not sampled, so those links are documented rather than source-verified.

Main entry points: bert.py BERT encoder · gpt.py GPT language model · bpe.py Byte-level BPE · transformer.py Seq2seq Transformer

Components 29

BERT classification

BERT corpus and data BERT/utils/data_utils.py
BERT pretraining BERT/scripts/pretrain.py
BERT encoder BERT/models/bert.py
Classification fine-tuning BERT/scripts/finetune.py
Labeled data batching BERT/utils/finetune_data.py
News classifier demo BERT/scripts/app.py
Classification inference BERT/scripts/inference.py

GPT-2 generation

Byte-level BPE GPT/bpe/bpe.py
Byte coding GPT/bpe/byte_coder.py
Pre-tokenization GPT/bpe/pretokenize.py
GPT corpus and data GPT/utils/data_utils.py
GPT pretraining GPT/scripts/pretrain.py
GPT language model GPT/models/gpt.py
Text generation GPT/scripts/generate.py

Transformer translation

Translation data transformer/utils/data_utils.py
Model training transformer/scripts/train.py
Seq2seq Transformer transformer/models/transformer.py
Translation inference transformer/scripts/inference.py

ViT images

Image data setup ViT/utils/data_setup.py
Dataset splitting ViT/utils/split_dataset.py
Image training ViT/utils/engine.py
Image predictions ViT/utils/predictions.py

Shared model layers

Attention and layers transformer/models/modules/multi_head_attention.py
Layer normalization transformer/models/modules/layer_norm.py

Other components

User
Bengali Wikipedia
IndicGLUE sna.bn
Samanantar corpus
Temple images

Connections 29

Bengali Wikipedia → BERT corpus and data: supplies corpus README.md
BERT corpus and data → BERT pretraining: prepares data
BERT pretraining → BERT encoder: trains
IndicGLUE sna.bn → Labeled data batching: supplies labels README.md
Labeled data batching → Classification fine-tuning: batches examples BERT/utils/finetune_data.py
Classification fine-tuning → BERT encoder: fine-tunes
User → News classifier demo: submits text BERT/scripts/app.py
News classifier demo → Classification inference: calls inference BERT/scripts/app.py
BERT encoder → Attention and layers: reuses attention
Bengali Wikipedia → GPT corpus and data: supplies corpus README.md
GPT corpus and data → GPT pretraining: prepares data
GPT pretraining → GPT language model: trains
GPT language model → Byte-level BPE: uses tokenizer
Byte-level BPE → Byte coding: encodes bytes GPT/bpe/bpe.py
Byte-level BPE → Pre-tokenization: splits text GPT/bpe/bpe.py
User → Text generation: requests generation
Text generation → GPT language model: generates with
Samanantar corpus → Translation data: supplies pairs README.md
Translation data → Model training: provides batches
Model training → Seq2seq Transformer: trains
User → Translation inference: requests translation
Translation inference → Seq2seq Transformer: decodes with
Seq2seq Transformer → Attention and layers: uses layers transformer/models/transformer.py
Temple images → Image data setup: supplies images README.md
Image data setup → Dataset splitting: prepares splits
Dataset splitting → Image training: provides datasets
Image training → Image predictions: produces predictions
BERT encoder → Layer normalization: uses normalization
GPT language model → Layer normalization: uses normalization GPT/models/gpt.py

Drawn by GitDiagram on November 12, 2026 from the repository's file tree, README and source files. Read as Markdown ·