How kunjcr2/how-llms-are-made works
Project type and purpose
• This repository is a monorepo of research code, notebooks, models, and documentation for large-language-model (LLM) experiments, including:
• DeepSeek: custom MoE-inspired attention modules and research papers
• GPT: end-to-end notebooks covering tokenization, attention, architecture, training, post-training, and fine-tuning
• Kimi: Mixture‐of‐D experts research modules
• MLandDL & Reasoning Models: classical ML/DL lecture notes and code
• My Models: stored fine-tuned model artifacts plus a minimal serving app (app.py)
• Purpose: Explore, implement, and compare LLM techniques (attention variants, MoE, fine-tuning) and serve a selected fine-tuned model.
File structure to infer architecture
• Top-level directories each represent a component or “logical service”:
• DeepSeek & Kimi: Python libraries/modules containing core algorithm implementations (.py) and PDFs/markdown for theory
• GPT: Jupyter notebooks driving data flow from raw text → tokenization → model building/training → fine-tuning
• My Models: model storage (safetensors, config, tokenizer) + app.py as a minimal model-serving layer
• MLandDL & Reasoning Models: supplemental educational material, not part of the runtime flow
• Configuration/build: No Dockerfile or CI scripts present; version control only.
• Deployment: Only My Models/app.py suggests a runtime service (likely Flask/FastAPI) that loads saved model.
Insights from README
• It merely points to the main topic folders; no high-level architecture specified.