# satyamdixit6/llm_ops architecture

> Architecture diagram of the GitHub repository satyamdixit6/llm_ops, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/satyamdixit6/llm_ops
- Repository: https://github.com/satyamdixit6/llm_ops

## Overview

##### Project Type and Purpose
- This is an open-source, modular LLM pipeline built with PyTorch & PyTorch Lightning.
- It provides a command-line orchestrator (`main.py`) for running various adaptation strategies (quantization, LoRA, RAG, FlashAttention, custom CUDA kernels) on large language models.
- It emphasizes efficient data handling, extensible adaptation modules, and reproducible experiment management.

##### High-Level File Structure & Architectural Patterns
- Top-level directories/files:
- `main.py` – Entry point / CLI orchestrator
- `base_llm_system.py` – Core LightningModule (model training/inference logic)
- `data_utils.py` / `model_utils.py` – Data loaders & model loaders
- `adaptations/` – One folder per optimization strategy
- `configs/` – YAML configuration files
- `llm_pipeline_output/` – Runtime outputs (logs, checkpoints)
- Pattern: a “pipeline” or “plugin” architecture. The orchestrator picks one adaptation, loads common utilities & the base LLM, then dynamically applies the chosen adapter.

##### Extracted Components
a. User / CLI Interface
- `main.py` (parses args, selects adaptation, kicks off the workflow)
b. Configuration Loader
- YAML files under `configs/` (optional load) + CLI arguments
c. Data Layer
- `data_utils.py`: tokenizer, dataset, dataloader
d. Model Layer
- `model_utils.py`: loading Hugging Face models
- `base_llm_system.py`: LightningModule encapsulating loss, optimizer, training loops
e. Adaptation Modules (plugins)
- quantization.py
- lora.py
- rag.py (includes document store & FAISS retriever)
- flash_attention.py
- custom_cuda_kernel/ (CUDA kernel ➔ kernel_wrapper.py)
f. Execution Engine
- PyTorch Lightning trainer, mixed precision, logging
g. External Services & Libraries
- FAISS (for RAG)
- PEFT (for LoRA)
- FlashAttention library or PyTorch SDPA
- CUDA Toolkit / nvcc for custom kernels
h. Output Store
- `llm_pipeline_output/` (logs, model checkpoints, quantized models, LoRA adapters)

##### Relationships & Data Flows
- CLI → Config Loader
- Config Loader → Data Layer & Model Layer
- Data Layer → Execution Engine (training/inference loops)
- Model Layer → Execution Engine
- Adaptation Layer → Execution Engine (each adapter injects hooks or replaces modules in the base system)
- Execution Engine → Output Store

##### Architectural Patterns & Design Principles
- Plugin architecture – each adaptation is a self-contained module under `adaptations/`.
- Single Responsibility – data_utils for data only, model_utils for loading only.
- Separation of concerns – core system vs. adaptation strategies.
- Extensibility – new adapters drop into `adaptations/` and register via `main.py`.

##### Technologies & Frameworks Call-Out
- PyTorch & PyTorch Lightning
- Hugging Face Transformers
- FAISS & sentence-transformers (for RAG)
- PEFT library (LoRA)
- FlashAttention (optional)
- Custom CUDA kernel via PyTorch extension (setup.py)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Interface
    subgraph "User Interface"
        CLI["User / CLI (main.py)"]:::core
    end

    %% Configuration Layer
    subgraph "Configuration Layer"
        Configs["configs/*.yaml + CLI flags"]:::data
    end

    %% Data Layer
    subgraph "Data Layer"
        Data["data_utils.py"]:::core
    end

    %% Model Layer
    subgraph "Model Layer"
        Utils["model_utils.py"]:::core
        Base["base_llm_system.py"]:::core
    end

    %% Adaptation Layer
    subgraph "Adaptation Layer"
        Q["quantization.py"]:::adaptation
        LORA["lora.py"]:::adaptation
        RAG["rag.py\n(SimpleDocumentStore)"]:::adaptation
        FA["flash_attention.py"]:::adaptation
        CUDA["custom_cuda_kernel"]:::adaptation
    end

    %% Execution Engine
    Execution["PyTorch Lightning Trainer\n+ mixed precision\n+ logging"]:::core

    %% Output Store
    subgraph "Output Store"
        Output["llm_pipeline_output/"]:::data
    end

    %% External Services
    subgraph "External Services"
        FAISS["FAISS"]:::external
        PEFT["PEFT"]:::external
        FlashLib["FlashAttention Library"]:::external
        CUDATool["CUDA Toolkit / nvcc"]:::external
    end

    %% Connections
    CLI -->|"args"| Configs
    Configs -->|"config"| Data
    Configs -->|"config"| Utils
    Data -->|"tokenized batches"| Execution
    Utils -->|"model weights"| Execution
    Base -->|"core logic"| Execution
    Q -->|"adapter hooks"| Execution
    LORA -->|"adapter hooks"| Execution
    RAG -->|"adapter hooks"| Execution
    FA -->|"adapter hooks"| Execution
    CUDA -->|"adapter hooks"| Execution
    Execution -->|"metrics & checkpoints"| Output

    %% RAG retrieval flow
    Execution -->|"retrieval query"| FAISS
    FAISS -->|"retrieved docs"| Execution

    %% External library usage
    LORA -->|"uses"| PEFT
    FA -->|"uses"| FlashLib
    CUDA -->|"uses"| CUDATool

    %% Click Events
    click CLI "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/main.py"
    click Configs "https://github.com/satyamdixit6/llm_ops/tree/main/llm_pipeline/configs/"
    click Data "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/data_utils.py"
    click Utils "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/model_utils.py"
    click Base "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/base_llm_system.py"
    click Q "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/adaptations/quantization.py"
    click LORA "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/adaptations/lora.py"
    click RAG "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/adaptations/rag.py"
    click FA "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/adaptations/flash_attention.py"
    click CUDA "https://github.com/satyamdixit6/llm_ops/tree/main/llm_pipeline/adaptations/custom_cuda_kernel/"
    click Execution "https://github.com/satyamdixit6/llm_ops/blob/main/llm_pipeline/base_llm_system.py"
    click Output "https://github.com/satyamdixit6/llm_ops/tree/main/llm_pipeline_output/"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef adaptation fill:#90EE90,stroke:#333,stroke-width:1px
    classDef data fill:#FFA500,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
```
