# nath1295/llmflex architecture

> Architecture diagram of the GitHub repository nath1295/llmflex, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nath1295/llmflex
- Repository: https://github.com/nath1295/llmflex

## Overview

##### Project type and purpose
- LLMFlex is an open-source Python library (CLI tool and Streamlit frontend) for building AI applications on local and remote large language models (LLMs).
- It provides modular abstractions for LLM loading/generation, embeddings, vector databases, tools, agents, memory, prompt templates, and a web UI.

##### File structure
- src/llmflex – the core library, organized into subpackages:
- Agents – orchestrators combining LLMs and tools
- Embeddings – embedding model toolkits
- Frontend – Streamlit interface and app resources
- KnowledgeBase – document loaders and retrieval wrappers
- Memory – chat memory classes (short-term, long-term via FAISS)
- Models – LLM “cores” (OpenAI, HuggingFace, llama-cpp, ExLlamaV2) + LlmFactory
- Prompts – PromptTemplate formatters
- Rankers – rerankers for search results
- Schemas – Pydantic document/token schemas
- TextSplitters – token-based text splitting
- Tools – BaseTool subclasses (browser tool, SD tool, web-search utils)
- VectorDBs – BaseVectordb and FaissVectorDatabase
- cli.py – entry point for CLI commands (interface, create-app-config)
- pyproject.toml – build/config metadata
- docs/ & docs/markdown – HTML/Markdown API documentation
- imgs – UI screenshots

##### README for additional insights
- Key dependencies: PyTorch, llama-cpp-python, optional ExLlamaV2, AutoGPTQ/AWQ, FAISS
- Two user front ends:
- CLI commands (e.g. llmflex interface)
- Streamlit webapp for chatting with memory, RAG, function calling
- Core features: LlmFactory, LangChain compatibility, embedding toolkits, RAG via Faiss, chat memories, agents, tools, prompt templates.

a. Main components
- User Interfaces: CLI layer, Streamlit Frontend
- Control/Orchestration: Agents, CLI dispatcher
- Model Layer: LlmFactory, Model Cores (OpenAICore, HuggingFaceCore, LlamaCppCore, ExllamaCore)
- Embedding Layer: EmbeddingsToolkit (HuggingFace/API), TextSplitters
- Data Layer: VectorDB (FaissVectorDatabase), Memory (BaseMemory, LongShortTermMemory), KnowledgeBase
- Tools Layer: BrowserTool, SDTool, ToolUtils, WebSearchUtils
- Prompt Layer: PromptTemplate
- External Services: OpenAI API servers, HuggingFace Hub, DuckDuckGo, local GPU drivers, FAISS native library

b. Relationships and interactions
- User → CLI or Streamlit → invokes CLI dispatcher/frontend controller
- Controller → selects/initializes LlmFactory → produces LLM instances
- Controller/Agent → uses PromptTemplate to format requests → calls LLM.invoke() or LLM.stream()
- Agent → orchestrates calls to Tools (e.g. BrowserTool) and Rankers, retrieving data
- EmbeddingsToolkit → generates embeddings for tools, KB, memory
- VectorDB/KnowledgeBase → store/query embeddings for RAG or long-term memory
- Memory → combines recent chat and long-term via VectorDB → returns context to Agent or Frontend
- Frontend → displays chat history, streams tokens, allows file uploads → updates KnowledgeBase

c. Architectural patterns & design principles
- Modular layered architecture: separation of concerns between UI, orchestration, modeling, data, tools
- Factory pattern: LlmFactory abstracts LLM instantiation/configuration
- Strategy pattern: interchangeable Cores, Embedding toolkits, Rankers, TextSplitters
- Adapter pattern: LangChain interface compatibility
- Plugin-style extensibility: BaseTool, BaseCore, BaseMemory, BaseVectordb

d. Relevant technologies/frameworks
- Python 3.x, PyTorch, llama-cpp-python, ExLlamaV2 (optional)
- FAISS for vector search
- Streamlit for UI
- Pydantic for schemas
- HTTP clients for API calls (OpenAI, HuggingFace, DuckDuckGo)

##### Putting it all together
1) UI Layer: CLI, Streamlit
2) Orchestration Layer: CLI Dispatcher, Agent, PromptTemplate
3) Core Services Layer:
- LlmFactory → Model Cores
- EmbeddingsToolkit → TextSplitters
- Tools → Rankers → WebSearch, SDTool
4) Data Layer: VectorDB, KnowledgeBase, Memory

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% UI Layer
    subgraph "UI Layer"
        direction TB
        CLI["CLI (llmflex)"]:::ui
        SI["Streamlit Interface"]:::ui
        SR["App Resource"]:::ui
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        direction TB
        Dispatcher["CLI Dispatcher"]:::orchestration
        Agent["Agent"]:::orchestration
        PromptTemplate["PromptTemplate"]:::orchestration
    end

    %% Core Services Layer
    subgraph "Core Services Layer"
        direction TB
        LLMFactory["LlmFactory"]:::core

        subgraph "Model Cores"
            direction TB
            BaseCore["BaseCore"]:::core
            OpenAICore["OpenAICore"]:::core
            HFCore["HuggingFaceCore"]:::core
            LlamaCPP["LlamaCppCore"]:::core
            ExLlama["ExLlamaV2Core"]:::core
            CoreUtils["CoreUtils"]:::core
        end

        subgraph "Embeddings Toolkit"
            direction TB
            APIEmb["APIEmbeddings"]:::core
            BaseEmb["BaseEmbeddings"]:::core
            HFServer["HFEmbeddingsServer"]:::core
            HFEmb["HuggingFaceEmbeddings"]:::core
        end

        subgraph "Text Splitters"
            direction TB
            BaseTS["BaseTextSplitter"]:::core
            SentenceTS["SentenceTokenTextSplitter"]:::core
            TokenTS["TokenTextSplitter"]:::core
        end

        subgraph "Tools & Rankers"
            direction TB
            BrowserTool["BrowserTool"]:::core
            SDTool["SDTool"]:::core
            ToolClasses["ToolClasses"]:::core
            ToolUtils["ToolUtils"]:::core
            WebSearch["WebSearchUtils"]:::core
            subgraph "Rankers"
                direction TB
                BaseRanker["BaseRanker"]:::core
                FlashRanker["FlashRankRanker"]:::core
            end
        end
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        subgraph "VectorDBs"
            direction TB
            BaseVDB["BaseVectordb"]:::data
            FaissVDB["FaissVectorDatabase"]:::data
        end
        subgraph "Memory"
            direction TB
            BaseMem["BaseMemory"]:::data
            LongShortMem["LongShortTermMemory"]:::data
            MemUtils["MemoryUtils"]:::data
        end
        KB["KnowledgeBase"]:::data
    end

    %% External Services
    subgraph "External Services"
        direction TB
        OpenAIAPI["OpenAI API"]:::external
        HFHub["HuggingFace Hub"]:::external
        DuckDuckGo["DuckDuckGo"]:::external
        GPU["Local GPU Drivers"]:::external
        FAISSLib["FAISS Native"]:::external
    end

    %% Connections
    CLI -->|invokes| Dispatcher
    SI -->|calls| Dispatcher
    Dispatcher -->|uses| LLMFactory
    Agent -->|uses| LLMFactory
    Dispatcher -->|formats| PromptTemplate
    Agent -->|formats| PromptTemplate
    PromptTemplate -->|generates prompt| OpenAICore
    PromptTemplate -->|generates prompt| HFCore
    PromptTemplate -->|generates prompt| LlamaCPP
    PromptTemplate -->|generates prompt| ExLlama

    LLMFactory -->|instantiates| BaseCore
    BaseCore --> OpenAICore
    BaseCore --> HFCore
    BaseCore --> LlamaCPP
    BaseCore --> ExLlama
    BaseCore --> CoreUtils

    BaseTS -->|splits| APIEmb
    BaseTS -->|splits| BaseEmb
    SentenceTS -->|splits| APIEmb
    TokenTS -->|splits| APIEmb

    APIEmb -->|embeds| BaseVDB
    BaseEmb -->|embeds| BaseVDB
    HFServer -->|embeds| BaseVDB
    HFEmb -->|embeds| BaseVDB

    BrowserTool -->|runs| DuckDuckGo
    SDTool -->|runs| GPU
    WebSearch -->|searches| DuckDuckGo

    BrowserTool -->|calls| FAISSLib
    SDTool -->|calls| FAISSLib

    BaseRanker -->|reranks| BaseVDB
    FlashRanker -->|reranks| BaseVDB

    BaseVDB -->|stores| KB
    FaissVDB -->|stores| KB

    KB -->|retrieves| Agent
    KB -->|retrieves| SI
    BaseMem -->|manages| Agent
    LongShortMem -->|manages| Agent
    MemUtils -->|helps| Agent
    FaissVDB -->|supports| LongShortMem

    OpenAICore -->|API call| OpenAIAPI
    HFCore -->|API call| HFHub
    LlamaCPP -->|local compute| GPU
    ExLlama -->|local compute| GPU

    %% Click Events
    click CLI "https://github.com/nath1295/llmflex/blob/master/src/llmflex/cli.py"
    click SI "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Frontend/streamlit_interface.py"
    click SR "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Frontend/app_resource.py"
    click Agent "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Agents/base_agent.py"
    click PromptTemplate "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Prompts/prompt_template.py"
    click LLMFactory "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Models/Factory/llm_factory.py"
    click OpenAICore "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Models/Cores/openai_core.py"
    click HFCore "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Models/Cores/huggingface_core.py"
    click LlamaCPP "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Models/Cores/llamacpp_core.py"
    click ExLlama "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Models/Cores/exllamav2_core.py"
    click BaseCore "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Models/Cores/base_core.py"
    click CoreUtils "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Models/Cores/utils.py"
    click APIEmb "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Embeddings/api_embeddings.py"
    click BaseEmb "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Embeddings/base_embeddings.py"
    click HFServer "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Embeddings/hf_embeddings_server.py"
    click HFEmb "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Embeddings/huggingface_embeddings.py"
    click BaseTS "https://github.com/nath1295/llmflex/blob/master/src/llmflex/TextSplitters/base_text_splitter.py"
    click SentenceTS "https://github.com/nath1295/llmflex/blob/master/src/llmflex/TextSplitters/sentence_token_text_splitter.py"
    click TokenTS "https://github.com/nath1295/llmflex/blob/master/src/llmflex/TextSplitters/token_text_splitter.py"
    click BaseVDB "https://github.com/nath1295/llmflex/blob/master/src/llmflex/VectorDBs/base_vectordb.py"
    click FaissVDB "https://github.com/nath1295/llmflex/blob/master/src/llmflex/VectorDBs/faiss_vectordb.py"
    click BaseMem "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Memory/base_memory.py"
    click LongShortMem "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Memory/long_short_memory.py"
    click MemUtils "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Memory/memory_utils.py"
    click KB "https://github.com/nath1295/llmflex/blob/master/src/llmflex/KnowledgeBase/knowledge_base.py"
    click BrowserTool "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Tools/browser_tool.py"
    click SDTool "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Tools/sd_tool.py"
    click ToolClasses "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Tools/tool_classes.py"
    click ToolUtils "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Tools/tool_utils.py"
    click WebSearch "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Tools/web_search_utils.py"
    click BaseRanker "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Rankers/base_ranker.py"
    click FlashRanker "https://github.com/nath1295/llmflex/blob/master/src/llmflex/Rankers/flashrank_ranker.py"

    %% Styles
    classDef ui fill:#E3F2FD,stroke:#1E88E5
    classDef orchestration fill:#E8F5E9,stroke:#43A047
    classDef core fill:#FFF3E0,stroke:#FB8C00
    classDef data fill:#FFFDE7,stroke:#FDD835
    classDef external fill:#FCE4EC,stroke:#D81B60
```
