How satyamdixit6/llm_ops works
Project Type and Purpose
• This is an open-source, modular LLM pipeline built with PyTorch & PyTorch Lightning.
• It provides a command-line orchestrator (main.py) for running various adaptation strategies (quantization, LoRA, RAG, FlashAttention, custom CUDA kernels) on large language models.
• It emphasizes efficient data handling, extensible adaptation modules, and reproducible experiment management.
High-Level File Structure & Architectural Patterns
• Top-level directories/files:
• main.py – Entry point / CLI orchestrator
• base_llm_system.py – Core LightningModule (model training/inference logic)
• data_utils.py / model_utils.py – Data loaders & model loaders
• adaptations/ – One folder per optimization strategy
• configs/ – YAML configuration files
• llm_pipeline_output/ – Runtime outputs (logs, checkpoints)
• Pattern: a “pipeline” or “plugin” architecture. The orchestrator picks one adaptation, loads common utilities & the base LLM, then dynamically applies the chosen adapter.
Extracted Components
a. User / CLI Interface
• main.py (parses args, selects adaptation, kicks off the workflow)
b. Configuration Loader
• YAML files under configs/ (optional load) + CLI arguments
c. Data Layer
• data_utils.py: tokenizer, dataset, dataloader
d. Model Layer
• model_utils.py: loading Hugging Face models
• base_llm_system.py: LightningModule encapsulating loss, optimizer, training loops
e. Adaptation Modules (plugins)
• quantization.py
• lora.py
• rag.py (includes document store & FAISS retriever)
• flash_attention.py
• custom_cuda_kernel/ (CUDA kernel ➔ kernel_wrapper.py)
f. Execution Engine
• PyTorch Lightning trainer, mixed precision, logging
g. External Services & Libraries
• FAISS (for RAG)
• PEFT (for LoRA)
• FlashAttention library or PyTorch SDPA
• CUDA Toolkit / nvcc for custom kernels
h. Output Store
• llm_pipeline_output/ (logs, model checkpoints, quantized models, LoRA adapters)
Relationships & Data Flows
• CLI → Config Loader
• Config Loader → Data Layer & Model Layer
• Data Layer → Execution Engine (training/inference loops)
• Model Layer → Execution Engine
• Adaptation Layer → Execution Engine (each adapter injects hooks or replaces modules in the base system)
• Execution Engine → Output Store
Architectural Patterns & Design Principles
• Plugin architecture – each adaptation is a self-contained module under adaptations/.
• Single Responsibility – data_utils for data only, model_utils for loading only.
• Separation of concerns – core system vs. adaptation strategies.
• Extensibility – new adapters drop into adaptations/ and register via main.py.
Technologies & Frameworks Call-Out
• PyTorch & PyTorch Lightning
• Hugging Face Transformers
• FAISS & sentence-transformers (for RAG)
• PEFT library (LoRA)
• FlashAttention (optional)
• Custom CUDA kernel via PyTorch extension (setup.py)