# aidatatools/llm_sentinel architecture

> Architecture diagram of the GitHub repository aidatatools/llm_sentinel, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aidatatools/llm_sentinel
- Repository: https://github.com/aidatatools/llm_sentinel

## Overview

##### Project Type and Purpose
- The project is a full-stack chatbot application focused on safeguarding LLM responses.
- It integrates various technologies for LLM moderation and safe outputs (using NeMo Guardrails and LangChain), along with a web interface provided by Gradio.
- The READMe indicates deployment concerns for both development (using a smaller llama3:8b model) and production (using llama3:70b). It also emphasizes running on NVIDIA GPUs, hinting at heavy LLM inference.

##### File Structure
- Top-level files include .env.example, README.md, LICENSE, and requirements.txt for environment and dependency setup.
- Multiple chatbot scripts (chatbot.py, chatbot2.py, chatbot3.py) suggest various entry points or experimental versions of chatbot behavior.
- The presence of guardrails_config.yaml indicates configuration for content moderation or safety rules, likely used in tandem with NeMo Guardrails.
- The img folder contains assets for documentation (e.g., setup screenshots), which can be ancillary to the system design.
- There are no separate frontend/backend directories, which implies a single-script based application that likely integrates both the web UI and the backend logic.

##### Key Architectural Components
- Web User Interface: Provided by Gradio (as noted in the READme), this component offers the browser-based interaction layer (endpoint http://127.0.0.1:7860).
- Chatbot Logic: The Python scripts (chatbot3.py as recommended for launching the chat interface) contain the core LLM interaction, including pre- and post-processing.
- LLM Inference Service: It relies on an ollama backend – an external service that hosts the LLM models (e.g., llama3:8b, llama3:70b) and processes inference requests. Communication likely happens over HTTP (as noted with the curl sample).
- Safe Guardrails: The project integrates NeMo Guardrails with its specific configuration in guardrails_config.yaml to monitor and moderate the LLM’s output.
- Configuration & Environment Management: The .env file (copied from .env.example) helps switch between production and development environments, while the requirements.txt and other configuration files ensure dependency management.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "User Interface"
        UI["Gradio UI"]:::frontend
    end

    subgraph "Core Logic"
        Chat["Chatbot Core Logic (chatbot3.py, chatbot.py, chatbot2.py)"]:::backend
    end

    subgraph "Configuration"
        Env["Environment (.env)"]:::config
        Dep["Dependencies (requirements.txt)"]:::config
    end

    subgraph "External Services"
        LLM["LLM Inference Backend (Ollama)"]:::external
        Mod["Content Moderation (NeMo Guardrails)"]:::moderation
    end

    UI -->|"processes"| Chat
    Env -->|"configures"| Chat
    Dep -->|"provides"| Chat
    Chat -->|"queries"| LLM
    Chat -->|"sends_output_for_moderation"| Mod
    LLM -->|"returns_response"| Chat
    Mod -->|"checks_and_flags"| Chat
    Chat -->|"responds_to"| UI

    click UI "https://github.com/aidatatools/llm_sentinel/blob/main/chatbot3.py"
    click Chat "https://github.com/aidatatools/llm_sentinel/blob/main/chatbot3.py"
    click Env "https://github.com/aidatatools/llm_sentinel/blob/main/.env.example"
    click Dep "https://github.com/aidatatools/llm_sentinel/blob/main/requirements.txt"
    click Mod "https://github.com/aidatatools/llm_sentinel/blob/main/guardrails_config.yaml"

    classDef frontend fill:#AED6F1,stroke:#000,stroke-width:2px;
    classDef backend fill:#A9DFBF,stroke:#000,stroke-width:2px;
    classDef config fill:#FAD7A0,stroke:#000,stroke-width:2px;
    classDef external fill:#D5F5E3,stroke:#000,stroke-width:2px;
    classDef moderation fill:#F5B7B1,stroke:#000,stroke-width:2px;
```
