# stevengonsalvez/llm-dojo architecture

> Architecture diagram of the GitHub repository stevengonsalvez/llm-dojo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/stevengonsalvez/llm-dojo
- Repository: https://github.com/stevengonsalvez/llm-dojo

## Overview

##### Main Components Organization

**┌─ Part 1: Understanding LLMs ─┐  ┌─ Part 2: Advanced Prompt Engineering ─┐**
- Models & Tokens           │  │ - Prompt Engineering Components       │
- Classification Systems    │  │ - Evaluation Systems                  │
- Chain of Thought         │  │ - N-shot & RAG Components            │
- ┘  └────────────────────────────────────────┘

**┌─ Part 3: Frameworks/Models ─┐  ┌─ Part 4: Beyond Text ─────────────────┐**
- Langchain Integration    │  │ - Diffusion Models                    │
- Model Implementations    │  │ - Training Pipeline                   │
- Platform Services        │  │ - Custom Data Processing              │
- ┘  └────────────────────────────────────────┘

##### Technical Components

a) Infrastructure Layer:
- Jupyter Notebook environments (both local and Google Colab)
- Environment management system (conda/virtualenv)
- Authentication & API key management

b) Core Components:
- GPT API Integration (from gpt-api directory)
- Langchain Framework (from langchain directory)
- Training Components (from training directory)
- Service Application (from apps/service-app)

##### Integration Points

- Data flow between notebooks and external APIs
- Environment variable management system
- Integration with Google Drive for Colab setup
- Connection to various LLM services (OpenAI, PALM2, etc.)

##### Specific Implementation Details

**┌─ Environment Management ─┐**
- Local .env            │
- Colab Drive Mount     │
- API Key Management    │
- ┘

**┌─ Notebook Environments ─┐**
- Local Jupyter        │
- Google Colab         │
- VSCode Integration   │
- ┘

##### Key Relationships to Highlight

- Data flow for training components
- Security boundaries around API key management
- Integration points with external frameworks

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Part 1: Understanding LLMs"
        Models["Models & Tokens"]
        Classification["Classification Systems"]
        COT["Chain of Thought"]
    end

    subgraph "Part 2: Advanced Prompt Engineering"
        PE["Prompt Engineering"]:::prompt
        Eval["Evaluation Systems"]:::eval
        RAG["RAG Components"]:::rag
        ReAct["ReAct Implementation"]:::react
    end

    subgraph "Part 3: Frameworks/Models"
        LC["Langchain Framework"]:::langchain
        MI["Model Implementations"]
        PS["Platform Services"]
        Agents["Agent Systems"]:::agents
        Memory["Memory Components"]:::memory
        Chains["Chain Components"]:::chains
        QA["Question-Answering"]:::qa
    end

    subgraph "Part 4: Beyond Text"
        DM["Diffusion Models"]
        TP["Training Pipeline"]:::training
        CDP["Custom Data Processing"]
    end

    subgraph "Infrastructure"
        ENV["Environment Management"]
        NB["Notebook Environments"]
        API["GPT API Integration"]:::gptapi
        SA["Service Application"]:::service
        GPT4["GPT-4 Functions"]:::gpt4
    end

    %% Relationships
    Models --> PE
    Classification --> Eval
    COT --> ReAct
    PE --> RAG
    PE --> ReAct
    LC --> Agents
    LC --> Memory
    LC --> Chains
    LC --> QA
    API --> SA
    API --> GPT4
    TP --> CDP
    ENV --> NB
    NB --> API

    %% Styling
    classDef prompt fill:#a8d5e5
    classDef eval fill:#d1c4e9
    classDef rag fill:#c8e6c9
    classDef react fill:#ffecb3
    classDef langchain fill:#b3e5fc
    classDef agents fill:#dcedc8
    classDef memory fill:#f8bbd0
    classDef chains fill:#d1c4e9
    classDef qa fill:#bbdefb
    classDef training fill:#ffccbc
    classDef gptapi fill:#e1bee7
    classDef service fill:#b2dfdb
    classDef gpt4 fill:#fff9c4

    %% Click Events
    click LC "https://github.com/stevengonsalvez/LLM-dojo/tree/main//langchain/"
    click API "https://github.com/stevengonsalvez/LLM-dojo/tree/main//gpt-api/"
    click TP "https://github.com/stevengonsalvez/LLM-dojo/tree/main//training/building-with-llms-and-chatgpt-api/"
    click SA "https://github.com/stevengonsalvez/LLM-dojo/tree/main//apps/service-app/"
    click RAG "https://github.com/stevengonsalvez/LLM-dojo/blob/main//misc/RAGFromScratch.ipynb"
    click ReAct "https://github.com/stevengonsalvez/LLM-dojo/blob/main//misc/ReActFromScratch.ipynb"
    click Eval "https://github.com/stevengonsalvez/LLM-dojo/blob/main//langchain/building/05-Evaluation.ipynb"
    click GPT4 "https://github.com/stevengonsalvez/LLM-dojo/blob/main//gen-ai/openai/gpt4/gpt-4-functions.ipynb"
    click PE "https://github.com/stevengonsalvez/LLM-dojo/blob/main//langchain/building/01_prompt_parsing.ipynb"
    click Memory "https://github.com/stevengonsalvez/LLM-dojo/blob/main//langchain/building/02-Memory.ipynb"
    click Chains "https://github.com/stevengonsalvez/LLM-dojo/blob/main//langchain/building/03-chains.ipynb"
    click QA "https://github.com/stevengonsalvez/LLM-dojo/blob/main//langchain/building/04-QnA.ipynb"
    click Agents "https://github.com/stevengonsalvez/LLM-dojo/blob/main//langchain/building/06-Agents.ipynb"
```
