# v2rockets/loyal-elephie architecture

> Architecture diagram of the GitHub repository v2rockets/loyal-elephie, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/v2rockets/loyal-elephie
- Repository: https://github.com/v2rockets/loyal-elephie

## Overview

##### Frontend Layer (Blue Components)
- Next.js Web Application
- Chat Interface
- Authentication System
- API Routes
- Connection to users.json for authentication

##### Backend Layer (Green Components)
- Python Flask Server (app.py)
- Memory Management System
- ChromaDB Integration
- BM25 Search Engine
- Language Processing
- NLTK Integration
- Multiple Language Support
- LLM Integration
- Model Interface
- XML Processing

##### External Services (Purple Components)
- LLM API (OpenAI-compatible)
- Embedding Service
- Optional Markdown Editor (SilverBulletMd)

##### Storage Components (Yellow Components)
- ChromaDB Vector Store
- Markdown Files Storage
- Chat History
- User Settings

- [User Browser] → [Frontend Layer] → [Backend Layer] → [External Services/Storage]

##### Bi-directional Data Flows
- User interactions ↔ Frontend
- Frontend ↔ Backend API calls
- Backend ↔ External Services
- Backend ↔ Storage Systems

##### Special Features
- Hybrid Search System (BM25 + ChromaDB)
- Memory Control Flow
- Multi-language Support
- Authentication Flow

Key Relationships to Highlight:

##### Frontend-Backend Communication
- WebSocket connections for real-time updates

##### Memory Management Flow
- Chroma_doc_manager.py to ChromaDB
- BM25 ranking system integration
- Memory persistence flow

##### Language Processing Pipeline
- NLTK integration with multiple language support
- To language_presets.json

##### External Integrations
- LLM API connection
- Markdown editor integration
- Embedding service connection

Styling Guidelines:

##### Labels
- Component names
- API endpoints
- Data flow descriptions
- Technology stack annotations

##### High-Level View
- Main system components
- Primary data flows
- External service connections

##### Detailed Subviews
- Memory management system
- Authentication flow
- Language processing pipeline
- Search system architecture

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client
        Browser["Web Browser"]:::client
    end

    subgraph FrontendLayer
        NextApp["Next.js Web App"]:::frontend
        click NextApp "https://github.com/v2rockets/Loyal-Elephie/tree/main/frontend"
        ChatUI["Chat Interface"]:::frontend
        click ChatUI "https://github.com/v2rockets/Loyal-Elephie/tree/main/frontend/components/Chat"
        AuthSystem["Authentication System"]:::frontend
        click AuthSystem "https://github.com/v2rockets/Loyal-Elephie/blob/main/frontend/pages/api/login.ts"
        APIRoutes["API Routes"]:::frontend
    end

    subgraph BackendLayer
        FlaskServer["Python Flask Server"]:::backend
        click FlaskServer "https://github.com/v2rockets/Loyal-Elephie/blob/main/backend/app.py"
        
        subgraph MemorySystem["Memory Management"]
            MemoryManager["Memory Manager"]:::backend
            click MemoryManager "https://github.com/v2rockets/Loyal-Elephie/blob/main/backend/memory_server.py"
            ChromaManager["ChromaDB Manager"]:::backend
            click ChromaManager "https://github.com/v2rockets/Loyal-Elephie/blob/main/backend/chroma_doc_manager.py"
            BM25["BM25 Search Engine"]:::backend
            click BM25 "https://github.com/v2rockets/Loyal-Elephie/blob/main/backend/bm25_api.py"
            FileMonitor["File Monitor"]:::backend
            click FileMonitor "https://github.com/v2rockets/Loyal-Elephie/blob/main/backend/file_monitor.py"
        end

        subgraph LanguageProcessing["Language Processing"]
            NLTK["NLTK Integration"]:::backend
            click NLTK "https://github.com/v2rockets/Loyal-Elephie/tree/main/backend/nltk_data"
            LangPresets["Language Presets"]:::backend
            click LangPresets "https://github.com/v2rockets/Loyal-Elephie/blob/main/backend/language_presets.json"
        end

        LLMInterface["LLM Interface"]:::backend
        click LLMInterface "https://github.com/v2rockets/Loyal-Elephie/blob/main/backend/llm_utils.py"
    end

    subgraph ExternalServices
        LLM_API["LLM API"]:::external
        EmbeddingService["Embedding Service"]:::external
        click EmbeddingService "https://github.com/v2rockets/Loyal-Elephie/blob/main/external_example/embedding_server.py"
        MarkdownEditor["SilverBulletMd Editor"]:::external
    end

    subgraph Storage
        UserAuth[("User Authentication")]:::storage
        click UserAuth "https://github.com/v2rockets/Loyal-Elephie/blob/main/frontend/users.json"
        ChromaDB[("ChromaDB Vector Store")]:::storage
        ChatHistory[("Chat History")]:::storage
        click ChatHistory "https://github.com/v2rockets/Loyal-Elephie/tree/main/md_website/chat_history"
        MarkdownFiles[("Markdown Files")]:::storage
        click MarkdownFiles "https://github.com/v2rockets/Loyal-Elephie/tree/main/md_website/notes"
        Settings[("System Settings")]:::storage
        click Settings "https://github.com/v2rockets/Loyal-Elephie/blob/main/md_website/SETTINGS.md"
    end

    %% Connections
    Browser <--> NextApp
    NextApp --> ChatUI
    NextApp --> AuthSystem
    NextApp --> APIRoutes
    APIRoutes <--> FlaskServer
    AuthSystem <--> UserAuth
    
    FlaskServer --> MemorySystem
    FlaskServer --> LanguageProcessing
    FlaskServer --> LLMInterface
    
    MemoryManager <--> ChromaDB
    MemoryManager <--> BM25
    ChromaManager <--> ChromaDB
    FileMonitor --> MarkdownFiles
    
    LLMInterface <--> LLM_API
    MemorySystem <--> EmbeddingService
    MarkdownEditor <--> MarkdownFiles
    
    NLTK --> LangPresets
    
    %% Styles
    classDef client fill:#f9f,stroke:#333,stroke-width:2px
    classDef frontend fill:#a8d1f0,stroke:#333,stroke-width:1px
    classDef backend fill:#90EE90,stroke:#333,stroke-width:1px
    classDef external fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef storage fill:#FFE4B5,stroke:#333,stroke-width:1px

    %% Legend
    subgraph Legend
        Client_["Client"]:::client
        Frontend_["Frontend"]:::frontend
        Backend_["Backend"]:::backend
        External_["External Services"]:::external
        Storage_["Storage"]:::storage
    end
```
