# peremartra/large-language-model-notebooks-course architecture

> Architecture diagram of the GitHub repository peremartra/large-language-model-notebooks-course, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/peremartra/large-language-model-notebooks-course
- Repository: https://github.com/peremartra/large-language-model-notebooks-course

## Overview

##### Project’s Nature and Scope
- The repository is organized as a course divided into three sections: Techniques & Libraries, Projects, and Enterprise Solutions.
- It contains many notebooks that demonstrate topics ranging from chatbots, vector databases, LangChain pipelines, evaluation metrics, fine tuning (via PEFT, LoRA, QLoRA), pruning methods, and even full project pipelines (NL2SQL and enterprise-grade solutions).
- The primary purpose is educational/demonstrative: it shows hands-on examples using OpenAI, Hugging Face, LangChain, and supporting tools.

##### Break Down the Main Components
a. Educational Modules and Lesson Components:
▪ Group them by the top-level directories (e.g., “1-Introduction to LLMs with OpenAI”, “2-Vector Databases with LLMs”, “3-LangChain”, “4-Evaluating LLMs”, “5-Fine Tuning”, “6-PRUNING”).
▪ Each of these sections represents a module where different techniques are explored.
b. Core Architectural Elements within Each Module:
▪ For techniques/libraries: Illustrate the connection between user input, prompt engineering, and the API/model call (OpenAI or open-source model).
▪ For vector databases: Represent the flow between datasets, embedding generation, and vector stores (e.g. ChromaDB, Faiss).
▪ For evaluation: Include components that send outputs from the language models to evaluation nodes (using BLEU, ROUGE, embedding similarity, LangSmith, etc.).
▪ For optimization (fine tuning and pruning): Show a feedback loop where the pre-trained base models are adapted using techniques like prompt tuning, LoRA, QLoRA, and then further refined via pruning methods (structured neuron pruning, attention layer pruning) and knowledge distillation.
c. Enterprise and Project-level Solutions:
▪ For Enterprise Solutions: Create a separate block showing integration points with legacy systems, the potential inclusion of vector databases for customer embeddings, and orchestration for transforming large-scale systems.

##### Architectural Patterns and Design Principles
- Microservices-like Components: Although not deployed as services, treat each notebook’s functionality (e.g., chatbot creation, NL2SQL, moderation, evaluation) as a mini-pipeline with clear inputs and outputs.
- Data Flow and Dependency Mapping: Clearly show dependencies, such as how the vector database provides context for the RAG system or how LangChain chains multiple calls.

##### Relevant Technologies and Libraries to Annotate
- APIs and Models: OpenAI GPT-3.5, Hugging Face models, LLAMA2, GPT-J.
- Tools and Frameworks: LangChain, PEFT, QLoRA, LoRA, ChromaDB, Faiss, lm-evaluation-harness, LangSmith.
- Evaluation Metrics: BLEU, ROUGE, embedding similarity.

## Diagram (Mermaid)

```mermaid
graph TD
    A["User Request"]:::user

    subgraph "Educational Modules"
        ED1["1-Introduction to LLMs with OpenAI"]:::edu
        PE1["1_3-Intro_Prompt_Engineering.ipynb"]:::edu
        PE2["Prompt_Engineering_OpenAI.ipynb"]:::edu

        ED2["2-Vector Databases with LLMs"]:::vec
        VDB1["2_2-ChromaDB Sever mode.ipynb"]:::vec
        VDB2["semantic_cache_chroma_vector_database.ipynb"]:::vec

        ED3["3-LangChain"]:::lang
        LC1["3_1_RAG_langchain.ipynb"]:::lang
        LC2["LangChain_Agent_create_Data_Scientist_Assistant.ipynb"]:::lang

        ED4["4-Evaluating LLMs"]:::eval
        EV1["4_1_bleu_evaluation.ipynb"]:::eval
        EV2["4_4_lm-evaluation-harness.ipynb"]:::eval

        ED5["5-Fine Tuning"]:::tune
        FT1["5_2_LoRA_Tuning.ipynb"]:::tune
        FT2["5_3_QLoRA_Tuning.ipynb"]:::tune
        FT3["Prompt_Tuning_PEFT.ipynb"]:::tune

        ED6["6-PRUNING"]:::prune
        PR1["6_1_pruning_structured_l1_diltilgpt2.ipynb"]:::prune
        PR2["6_6_pruning_attention_layers.ipynb"]:::prune
        PR3["7_1_knowledge_distillation_Llama.ipynb"]:::prune
    end

    subgraph "NL2SQL Projects"
        NL1["P1-NL2SQL"]:::proj
        NL2["E1-NL2SQL for big Databases"]:::proj
    end

    subgraph "Enterprise Solutions"
        ENT1["E2-Transforming Banks With Embeddings"]:::ent
        ENT2["P2-MHF"]:::ent
        ENT3["P3-CustomFinancialLLM"]:::ent
    end

    A -->|"triggers"| ED1
    A -->|"triggers"| ED2
    A -->|"triggers"| ED3
    A -->|"triggers"| ED4
    A -->|"triggers"| ED5
    A -->|"triggers"| ED6
    A -->|"triggers"| NL1
    A -->|"triggers"| NL2
    A -->|"triggers"| ENT1
    A -->|"triggers"| ENT2
    A -->|"triggers"| ENT3

    %% Link notebooks to their educational modules
    ED1 --> PE1
    ED1 --> PE2

    ED2 --> VDB1
    ED2 --> VDB2

    ED3 --> LC1
    ED3 --> LC2

    ED4 --> EV1
    ED4 --> EV2

    ED5 --> FT1
    ED5 --> FT2
    ED5 --> FT3

    ED6 --> PR1
    ED6 --> PR2
    ED6 --> PR3

    %% Data Flow Connections within Educational Modules
    PE1 -->|"providesPrompt"| ED3
    PE2 -->|"enhancesPrompt"| ED3
    VDB1 -->|"suppliesEmbeddings"| ED3
    VDB2 -->|"suppliesEmbeddings"| ED3
    LC1 -->|"processesData"| ED4
    LC2 -->|"handlesLogic"| ED4
    EV1 -->|"evaluatesOutput"| ED3
    EV2 -->|"evaluatesOutput"| ED3
    FT1 -->|"tunesModel"| ED3
    FT2 -->|"tunesModel"| ED3
    FT3 -->|"adjustsParameters"| ED3
    PR1 -->|"prunesModel"| ED3
    PR2 -->|"prunesModel"| ED3
    PR3 -->|"distillsKnowledge"| ED3

    %% Feedback Loop between Fine Tuning and Pruning
    ED5 -->|"feedback"| ED6
    ED6 -->|"refinedModel"| ED5

    %% Style Definitions
    classDef user fill:#f9f,stroke:#333,stroke-width:2px;
    classDef edu fill:#d1e8ff,stroke:#006,stroke-width:2px;
    classDef vec fill:#cce8cc,stroke:#008000,stroke-width:2px;
    classDef lang fill:#fde2e4,stroke:#d00000,stroke-width:2px;
    classDef eval fill:#fff2cc,stroke:#d4ac0d,stroke-width:2px;
    classDef tune fill:#eaffd0,stroke:#28a745,stroke-width:2px;
    classDef prune fill:#f0d9ff,stroke:#9b59b6,stroke-width:2px;
    classDef proj fill:#d0e6a5,stroke:#7dcea0,stroke-width:2px;
    classDef ent fill:#ffd6a5,stroke:#e67e22,stroke-width:2px;

    %% Click Events for Directories
    click ED1 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/1-Introduction to LLMs with OpenAI"
    click ED2 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/2-Vector Databases with LLMs"
    click ED3 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/3-LangChain"
    click ED4 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/4-Evaluating LLMs"
    click ED5 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/5-Fine Tuning"
    click ED6 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/6-PRUNING"
    click NL1 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/P1-NL2SQL"
    click NL2 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/E1-NL2SQL for big Databases"
    click ENT1 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/E2-Transforming Banks With Embeddings"
    click ENT2 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/P2-MHF"
    click ENT3 "https://github.com/peremartra/large-language-model-notebooks-course/tree/main/P3-CustomFinancialLLM"

    %% Click Events for Files/Notebooks
    click PE1 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/1-Introduction to LLMs with OpenAI/1_3-Intro_Prompt_Engineering.ipynb"
    click PE2 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/1-Introduction to LLMs with OpenAI/Prompt_Engineering_OpenAI.ipynb"
    click VDB1 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/2-Vector Databases with LLMs/2_2-ChromaDB Sever mode.ipynb"
    click VDB2 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/2-Vector Databases with LLMs/semantic_cache_chroma_vector_database.ipynb"
    click LC1 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/3-LangChain/3_1_RAG_langchain.ipynb"
    click LC2 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/3-LangChain/LangChain_Agent_create_Data_Scientist_Assistant.ipynb"
    click EV1 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/4-Evaluating LLMs/4_1_bleu_evaluation.ipynb"
    click EV2 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/4-Evaluating LLMs/4_4_lm-evaluation-harness.ipynb"
    click FT1 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/5-Fine Tuning/5_2_LoRA_Tuning.ipynb"
    click FT2 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/5-Fine Tuning/5_3_QLoRA_Tuning.ipynb"
    click FT3 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/5-Fine Tuning/Prompt_Tuning_PEFT.ipynb"
    click PR1 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/6-PRUNING/6_1_pruning_structured_l1_diltilgpt2.ipynb"
    click PR2 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/6-PRUNING/6_6_pruning_attention_layers.ipynb"
    click PR3 "https://github.com/peremartra/large-language-model-notebooks-course/blob/main/6-PRUNING/7_1_knowledge_distillation_Llama.ipynb"
```
