# sreejanpersonal/jarvis-agi architecture

> Architecture diagram of the GitHub repository sreejanpersonal/jarvis-agi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sreejanpersonal/jarvis-agi
- Repository: https://github.com/sreejanpersonal/jarvis-agi

## Overview

##### Project type and purpose
- This is a full-stack AI assistant ("JARVIS-AGI") built in Python.
- Purpose: accept voice/text/image input, dispatch to speech-to-text (STT), natural-language/image-AI engines, then produce text responses and speech output.

##### High-level layered architecture
- User I/O layer
– Voice (microphone) and text (CLI/terminal) inputs
– Audio playback and console output
- Engine layer
– STT subsystem (Engine/STT: Vosk, DevsDoCode, NetHyTech)
– TTS subsystem (Engine/TTS: ElevenLabs, DeepGram, streaming variants)
- Brain (Core AI) layer
– Text-AI (BRAIN/AI/TEXT: multiple API adapters, local llama_CPP, streaming)
– Vision-AI (BRAIN/AI/VISION: deepInfra_VISION)
– Image-AI (BRAIN/AI/IMAGE modules)
– Tools (BRAIN/TOOLS and TOOLS/*: utilities like groq_web_access, Hotword_Detection, system settings)
- Assets & Data layer
– ASSETS: model files (Vosk, clap detection), sounds, user data, proxies, conversation logs
- Orchestrator
– main.py + IMPORTS.py glues layers together, boots config (.env), reads assets, invokes STT → Brain → TTS pipeline.

##### Key components and interactions
- User
→ sends audio/text to Orchestrator (main.py)
- STT Engine
← reads model from ASSETS/Vosk
→ emits text to Brain
- Brain
– Text branch: picks one of TEXT/API connectors or LOCAL/STREAM modules, optionally calls external APIs (OpenAI, HuggingFace, FarFalle, etc.)
– Vision/Image branch: processes image inputs via deepInfra_IMG, decohere_ai
– Tools branch: auxiliary services (web search, hotword detection)
← optionally loads prompts from PROMPTS/
→ returns AI-generated text
- TTS Engine
← receives text, invokes chosen provider (ElevenLabs, Edge-TTS, speechify, DeepGram)
→ outputs audio to user
- Assets & Data
– Models and config loaded at startup
– conversation_history.json updated by orchestrator
– openGPT_IDs.txt, available_working_proxies.txt for external API management

##### Architectural patterns and design principles
- Layered pipeline: I/O → STT → AI Brain → TTS → I/O
- Plugin/Adapter pattern: each AI service is an interchangeable Python module under BRAIN/AI/TEXT/API or ENGINE/TTS/STREAMING
- Configuration-driven: .env for API keys and model paths
- Separation of concerns:
– ENGINE modules know only about speech/audio,
– BRAIN modules handle AI logic,
– TOOLS hold utilities,
– PLAYGROUND for experiments (not core at runtime).

##### Capturing essential elements
- Main pipelines and data flows
- Important libraries/frameworks:
– Vosk (STT), ElevenLabs/DeepGram (TTS), OpenAI/HuggingFace (Text AI), llama_CPP (local AI)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User I/O Layer
    subgraph "User I/O Layer"
        direction TB
        User["User (Voice/Text)"]:::external
        Output["Audio Playback / Console Output"]:::external
    end

    %% Orchestrator
    subgraph "Orchestrator"
        direction TB
        Main["main.py"]:::core
        Imports["IMPORTS.py"]:::core
        Env[".env.example"]:::core
    end

    %% Engine Layer
    subgraph "Engine Layer"
        direction TB
        subgraph "STT Engines" 
            direction TB
            VoskRecog["Vosk"]:::core
            DevsDoCode["DevsDoCode"]:::core
            NetHyTech["NetHyTech"]:::core
        end
        subgraph "TTS Engines"
            direction TB
            ElevenLabs["ElevenLabs"]:::core
            DeepGramTTS["DeepGram"]:::core
            EdgeTTS["edge_tts"]:::core
            DeepAI["deepAI"]:::core
            AIVoice["ai_voice"]:::core
            Speechify["speechify"]:::core
            subgraph "Streaming TTS"
                direction TB
                StreamDeepGram["DeepGram (Streaming)"]:::core
                StreamSpeechify["speechify (Streaming)"]:::core
                StreamElements["stream_elements_api"]:::core
            end
        end
    end

    %% Brain Layer
    subgraph "Brain (Core AI)"
        direction TB
        subgraph "Text AI"
            direction TB
            subgraph "API Adapters"
                direction TB
                TextAPI["Adapters"]:::core
            end
            subgraph "Local Model"
                direction TB
                LlamaCPP["llama_CPP"]:::core
            end
            subgraph "Streaming Text"
                direction TB
                BasedGPT["basedGPT"]:::core
                DeepInfraText["deepInfra_TEXT"]:::core
            end
        end
        subgraph "Vision AI"
            direction TB
            DeepInfraVision["deepInfra_VISION"]:::core
        end
        subgraph "Image AI"
            direction TB
            DecohereAI["decohere_ai"]:::core
            DeepInfraIMG["deepInfra_IMG"]:::core
        end
        subgraph "Brain Tools"
            direction TB
            GroqWeb["groq_web_access"]:::core
        end
    end

    %% Tools Layer
    subgraph "General Tools"
        direction TB
        Hotword["Hotword_Detection"]:::core
        Interrupted["Interrupted_Playsound"]:::core
        ProxyAPI["ProxyAPI"]:::core
        WebResults["Web_Results"]:::core
        subgraph "System Settings"
            direction TB
            Alarms["Alarms"]:::core
            Setting["SETTING"]:::core
            Theme["system_theme"]:::core
            Taskbar["taskbar"]:::core
        end
    end

    %% Assets & Data
    subgraph "Assets & Data"
        direction TB
        VoskModel["Vosk Model Files"]:::data
        ClapModel["CLAP_DETECTS Model.txt"]:::data
        Sounds["Sounds Folder"]:::data
        UserData["User Data"]:::data
        Proxies["available_working_proxies.txt"]:::data
        History["conversation_history.json"]:::data
        OpenGPTIDs["openGPT_IDs.txt"]:::data
    end

    %% Prompts
    subgraph "Prompts"
        direction TB
        PromptsFolder["PROMPTS Scripts"]:::core
    end

    %% External Services
    subgraph "External AI Services"
        direction TB
        OpenAI["OpenAI"]:::external
        HuggingFace["HuggingFace"]:::external
        FarFalle["FarFalle"]:::external
        BlackboxAI["Blackbox_ai"]:::external
    end

    %% Connections
    User -->|"input"| Main
    Main -->|"- startup loads env"| Env
    Main -->|"- import modules"| Imports
    Main -->|"invoke"| VoskRecog
    Main -->|"invoke"| DevsDoCode
    Main -->|"invoke"| NetHyTech
    VoskRecog -->|"transcribed text"| TextAPI
    DevsDoCode --> TextAPI
    NetHyTech --> TextAPI
    TextAPI --> LlamaCPP
    TextAPI --> BasedGPT
    TextAPI --> DeepInfraText
    TextAPI --> OpenAI
    TextAPI --> HuggingFace
    TextAPI --> FarFalle
    TextAPI --> BlackboxAI
    LlamaCPP --> TextAPI
    BasedGPT --> TextAPI
    DeepInfraText --> TextAPI
    TextAPI --> GroqWeb
    PromptsFolder --> TextAPI
    DeepInfraVision --> GroqWeb
    DecohereAI --> DeepInfraVision
    DeepInfraIMG --> DeepInfraVision
    TextAPI -->|"response text"| ElevenLabs
    TextAPI --> DeepGramTTS
    TextAPI --> EdgeTTS
    TextAPI --> DeepAI
    TextAPI --> AIVoice
    TextAPI --> Speechify
    ElevenLabs --> Output
    DeepGramTTS --> Output
    EdgeTTS --> Output
    DeepAI --> Output
    AIVoice --> Output
    Speechify --> Output
    StreamDeepGram --> Output
    StreamSpeechify --> Output
    StreamElements --> Output

    Main --> VoskModel
    VoskRecog --> VoskModel
    GroqWeb --> Proxies
    Main --> History
    Main --> OpenGPTIDs
    Main --> Sounds
    Main --> UserData

    %% Click Events
    click Main "https://github.com/sreejanpersonal/jarvis-agi/blob/main/main.py"
    click Imports "https://github.com/sreejanpersonal/jarvis-agi/blob/main/IMPORTS.py"
    click Env "https://github.com/sreejanpersonal/jarvis-agi/blob/main/.env.example"
    click VoskRecog "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/STT/vosk_recog.py"
    click DevsDoCode "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/STT/DevsDoCode.py"
    click NetHyTech "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/STT/NetHyTech.py"
    click VoskModel "https://github.com/sreejanpersonal/jarvis-agi/tree/main/ASSETS/Vosk/vosk-model-small-en-us-0.15/"
    click ElevenLabs "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/ElevenLabs.py"
    click DeepGramTTS "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/DeepGram.py"
    click EdgeTTS "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/edge_tts.py"
    click DeepAI "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/deepAI.py"
    click AIVoice "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/ai_voice.py"
    click Speechify "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/speechify.py"
    click StreamDeepGram "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/STREAMING/DeepGram.py"
    click StreamSpeechify "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/STREAMING/speechify.py"
    click StreamElements "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ENGINE/TTS/STREAMING/stream_elements_api.py"
    click TextAPI "https://github.com/sreejanpersonal/jarvis-agi/tree/main/BRAIN/AI/TEXT/API/"
    click LlamaCPP "https://github.com/sreejanpersonal/jarvis-agi/blob/main/BRAIN/AI/TEXT/LOCAL/llama_CPP.py"
    click BasedGPT "https://github.com/sreejanpersonal/jarvis-agi/blob/main/BRAIN/AI/TEXT/STREAM/basedGPT.py"
    click DeepInfraText "https://github.com/sreejanpersonal/jarvis-agi/blob/main/BRAIN/AI/TEXT/STREAM/deepInfra_TEXT.py"
    click DeepInfraVision "https://github.com/sreejanpersonal/jarvis-agi/blob/main/BRAIN/AI/VISION/deepInfra_VISION.py"
    click DecohereAI "https://github.com/sreejanpersonal/jarvis-agi/blob/main/BRAIN/AI/IMAGE/decohere_ai.py"
    click DeepInfraIMG "https://github.com/sreejanpersonal/jarvis-agi/blob/main/BRAIN/AI/IMAGE/deepInfra_IMG.py"
    click GroqWeb "https://github.com/sreejanpersonal/jarvis-agi/blob/main/BRAIN/TOOLS/groq_web_access.py"
    click Hotword "https://github.com/sreejanpersonal/jarvis-agi/blob/main/TOOLS/AUDIO/Hotword_Detection.py"
    click Interrupted "https://github.com/sreejanpersonal/jarvis-agi/blob/main/TOOLS/AUDIO/Interrupted_Playsound.py"
    click ProxyAPI "https://github.com/sreejanpersonal/jarvis-agi/blob/main/TOOLS/ProxyAPI.py"
    click WebResults "https://github.com/sreejanpersonal/jarvis-agi/blob/main/TOOLS/Web_Results.py"
    click Alarms "https://github.com/sreejanpersonal/jarvis-agi/blob/main/TOOLS/SYSTEM_SETTINGS/Alarms.py"
    click Setting "https://github.com/sreejanpersonal/jarvis-agi/blob/main/TOOLS/SYSTEM_SETTINGS/SETTING.py"
    click Theme "https://github.com/sreejanpersonal/jarvis-agi/blob/main/TOOLS/SYSTEM_SETTINGS/system_theme.py"
    click Taskbar "https://github.com/sreejanpersonal/jarvis-agi/blob/main/TOOLS/SYSTEM_SETTINGS/taskbar.py"
    click ClapModel "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ASSETS/CLAP_DETECTS/MODELS/Model.txt"
    click Sounds "https://github.com/sreejanpersonal/jarvis-agi/tree/main/ASSETS/SOUNDS/"
    click UserData "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ASSETS/USERDATA/LE CHAT/How_To_Store_UserData.txt"
    click Proxies "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ASSETS/available_working_proxies.txt"
    click History "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ASSETS/conversation_history.json"
    click OpenGPTIDs "https://github.com/sreejanpersonal/jarvis-agi/blob/main/ASSETS/openGPT_IDs.txt"
    click PromptsFolder "https://github.com/sreejanpersonal/jarvis-agi/tree/main/PROMPTS/"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef data fill:#90EE90,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
```
