# harish-nesrekar/vyom architecture

> Architecture diagram of the GitHub repository harish-nesrekar/vyom, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harish-nesrekar/vyom
- Repository: https://github.com/harish-nesrekar/vyom

## Overview

##### Project type and purpose
- From the README (“VYOM: Enhancing OpenAI’s GPT-OSS with Multimodal Vision Capabilities extensible to ISRO EO Data”) and the mix of Next.js, Express/Python scripts, this is a full-stack, AI-powered web application.
- Frontend: Next.js-based UI for uploading images, viewing maps, chat interface, result cards.
- Backend: Node.js/Express API server (server.js, aiController.js, nasaController.js, authcontroller.js), plus Python-based microservices for vision/EO analysis (blip_loader.py, eo_analysis.py, vqa.py, risk_analysis.py, etc.).

##### File structure
- Top-level UI components (AuthContext.js, ChatBox.js, EOMapViewer.js, NavBar.js, etc.) and global styling (globals.css).
- `frontend/` folder: Next.js app directory (app/page.js, app/layout.js, public assets), ESLint, PostCSS configs.
- Top-level Node code: server.js, api.js, ai.js, auth.js, authMiddleware.js – core Express routes, middleware, and API orchestration.
- Python scripts at root: modular microservices for loading models, running inference, generating captions, doing EO analysis, training (`train_eurosat.py`), evaluation.
- `backend/` folder: legacy or additional Node modules (unclear – may house shared utilities or a separate microservice).

##### README for additional insights
- Extensibility to ISRO EO Data suggests pluggable loader components (e.g., nasa.js, nasaController.js, eurosat_loader.py).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Layer
    User["User"]:::external

    %% Frontend Layer
    subgraph "Frontend Layer"
        direction TB
        NextJS["Next.js Frontend"]:::frontend
        AuthContext["AuthContext.js"]:::frontend
        ChatBox["ChatBox.js"]:::frontend
        EOMapViewer["EOMapViewer.js"]:::frontend
        ImageUploader["ImageUploader.js"]:::frontend
        NavBar["NavBar.js"]:::frontend
        ResultCard["ResultCard.js"]:::frontend
        AutoSatAnalysis["AutoSatelliteAnalysis.js"]:::frontend
    end

    %% Backend API Layer
    subgraph "Backend API Layer"
        direction TB
        Server["server.js"]:::backend
        API["api.js"]:::backend
        AIJS["ai.js"]:::backend
        AuthJS["auth.js"]:::backend
        subgraph Controllers
            direction TB
            AIController["aiController.js"]:::backend
            NASAController["nasaController.js"]:::backend
            AuthController["authcontroller.js"]:::backend
        end
        AuthMiddleware["authMiddleware.js"]:::backend
        NASAHelper["nasa.js"]:::backend
        NASAFetch["nasaFetch.js"]:::backend
    end

    %% Python Microservices Layer
    subgraph "Python Microservices Layer"
        direction TB
        APIServerPy["api_server.py"]:::python
        subgraph VisionLoaders
            direction TB
            BLIPLoader["blip_loader.py"]:::python
            Caption["caption.py"]:::python
            GPTOSSLoader["gpt_oss_loader.py"]:::python
            LLAVALoader["llava_loader.py"]:::python
        end
        subgraph EOAnalysis
            direction TB
            EOAnalysis["eo_analysis.py"]:::python
            EuroSatLoader["eurosat_loader.py"]:::python
            EvalEuroSat["evaluate_eurosat.py"]:::python
            TrainEuroSat["train_eurosat.py"]:::python
        end
        subgraph QA_Translation
            direction TB
            VQA["vqa.py"]:::python
            Translate["translate.py"]:::python
            TranslatorJS["translator.js"]:::python
        end
        subgraph RiskReport
            direction TB
            RiskAnalysis["risk_analysis.py"]:::python
            ReportGen["report_generator.py"]:::python
        end
    end

    %% Database Layer
    subgraph "Database / User Store"
        direction TB
        UserModel["User.js"]:::db
        ConfigDB["config.json"]:::db
    end

    %% Config & Model Artifacts
    subgraph "Config & Model Artifacts"
        direction TB
        TokenizerJSON["tokenizer.json"]:::db
        TokenizerCfg["tokenizer_config.json"]:::db
        SpecialTokens["special_tokens_map.json"]:::db
        SpieceModel["spiece.model"]:::db
        SafetensorsIdx["model.safetensors.index.json"]:::db
        ProcessorCfg["processor_config.json"]:::db
        PreprocessorCfg["preprocessor_config.json"]:::db
        GenerationCfg["generation_config.json"]:::db
    end

    %% Legacy Backend Modules
    subgraph "Legacy / Additional Backend"
        direction TB
        LegacyPkg["backend/package.json"]:::backend
        LegacyLock["backend/package-lock.json"]:::backend
        LegacyNodeMods["backend/node_modules"]:::backend
    end

    %% External Services
    subgraph "External Services"
        direction TB
        OpenAI["OpenAI GPT-OSS API"]:::external
        NASA["NASA EO API"]:::external
    end

    %% Connections
    User -->|Uses| NextJS
    NextJS -->|HTTP/REST| Server
    Server -->|routes through| API
    Server -->|uses| AIJS
    Server -->|uses| AuthJS
    API -->|invokes| AIController
    API -->|invokes| NASAController
    API -->|invokes| AuthController
    AIController -->|spawns/requests| APIServerPy
    NASAController -->|calls| NASAHelper
    NASAHelper -->|fetches| NASAFetch
    AuthController -->|checks| AuthMiddleware
    AuthMiddleware -->|reads/writes| UserModel
    Server -->|reads| ConfigDB
    APIServerPy -->|loads models from| SpieceModel
    APIServerPy -->|tokenizes via| TokenizerJSON
    APIServerPy -->|uses| ProcessorCfg
    BLIPLoader --> Caption
    GPTOSSLoader --> Caption
    LLAVALoader --> Caption
    Caption -->|generates| EOAnalysis
    EOAnalysis -->|loads data via| EuroSatLoader
    EvalEuroSat -->|evaluates| EOAnalysis
    TrainEuroSat -->|trains model| EuroSatLoader
    VQA -->|answers| EOAnalysis
    Translate -->|translates| VQA
    TranslatorJS --> Translate
    RiskAnalysis --> ReportGen
    Server -->|connects to| UserModel
    AIController -->|calls| OpenAI
    NASAController -->|calls| NASA
    NextJS -->|renders| EOMapViewer
    NextJS -->|renders| ResultCard
    NextJS -->|renders| ChatBox
    NextJS -->|renders| ImageUploader
    NextJS -->|renders| NavBar
    NextJS -->|renders| AuthContext
    NextJS -->|renders| AutoSatAnalysis

    %% Click Events
    click NextJS "https://github.com/harish-nesrekar/vyom/tree/main/frontend/app"
    click AuthContext "https://github.com/harish-nesrekar/vyom/blob/main/AuthContext.js"
    click ChatBox "https://github.com/harish-nesrekar/vyom/blob/main/ChatBox.js"
    click EOMapViewer "https://github.com/harish-nesrekar/vyom/blob/main/EOMapViewer.js"
    click ImageUploader "https://github.com/harish-nesrekar/vyom/blob/main/ImageUploader.js"
    click NavBar "https://github.com/harish-nesrekar/vyom/blob/main/NavBar.js"
    click ResultCard "https://github.com/harish-nesrekar/vyom/blob/main/ResultCard.js"
    click AutoSatAnalysis "https://github.com/harish-nesrekar/vyom/blob/main/AutoSatelliteAnalysis.js"
    click Server "https://github.com/harish-nesrekar/vyom/blob/main/server.js"
    click API "https://github.com/harish-nesrekar/vyom/blob/main/api.js"
    click AIJS "https://github.com/harish-nesrekar/vyom/blob/main/ai.js"
    click AuthJS "https://github.com/harish-nesrekar/vyom/blob/main/auth.js"
    click AIController "https://github.com/harish-nesrekar/vyom/blob/main/aiController.js"
    click NASAController "https://github.com/harish-nesrekar/vyom/blob/main/nasaController.js"
    click AuthController "https://github.com/harish-nesrekar/vyom/blob/main/authcontroller.js"
    click AuthMiddleware "https://github.com/harish-nesrekar/vyom/blob/main/authMiddleware.js"
    click NASAHelper "https://github.com/harish-nesrekar/vyom/blob/main/nasa.js"
    click NASAFetch "https://github.com/harish-nesrekar/vyom/blob/main/nasaFetch.js"
    click APIServerPy "https://github.com/harish-nesrekar/vyom/blob/main/api_server.py"
    click BLIPLoader "https://github.com/harish-nesrekar/vyom/blob/main/blip_loader.py"
    click Caption "https://github.com/harish-nesrekar/vyom/blob/main/caption.py"
    click GPTOSSLoader "https://github.com/harish-nesrekar/vyom/blob/main/gpt_oss_loader.py"
    click LLAVALoader "https://github.com/harish-nesrekar/vyom/blob/main/llava_loader.py"
    click EOAnalysis "https://github.com/harish-nesrekar/vyom/blob/main/eo_analysis.py"
    click EuroSatLoader "https://github.com/harish-nesrekar/vyom/blob/main/eurosat_loader.py"
    click EvalEuroSat "https://github.com/harish-nesrekar/vyom/blob/main/evaluate_eurosat.py"
    click TrainEuroSat "https://github.com/harish-nesrekar/vyom/blob/main/train_eurosat.py"
    click VQA "https://github.com/harish-nesrekar/vyom/blob/main/vqa.py"
    click Translate "https://github.com/harish-nesrekar/vyom/blob/main/translate.py"
    click TranslatorJS "https://github.com/harish-nesrekar/vyom/blob/main/translator.js"
    click RiskAnalysis "https://github.com/harish-nesrekar/vyom/blob/main/risk_analysis.py"
    click ReportGen "https://github.com/harish-nesrekar/vyom/blob/main/report_generator.py"
    click UserModel "https://github.com/harish-nesrekar/vyom/blob/main/User.js"
    click ConfigDB "https://github.com/harish-nesrekar/vyom/blob/main/config.json"
    click TokenizerJSON "https://github.com/harish-nesrekar/vyom/blob/main/tokenizer.json"
    click TokenizerCfg "https://github.com/harish-nesrekar/vyom/blob/main/tokenizer_config.json"
    click SpecialTokens "https://github.com/harish-nesrekar/vyom/blob/main/special_tokens_map.json"
    click SpieceModel "https://github.com/harish-nesrekar/vyom/blob/main/spiece.model"
    click SafetensorsIdx "https://github.com/harish-nesrekar/vyom/blob/main/model.safetensors.index.json"
    click ProcessorCfg "https://github.com/harish-nesrekar/vyom/blob/main/processor_config.json"
    click PreprocessorCfg "https://github.com/harish-nesrekar/vyom/blob/main/preprocessor_config.json"
    click GenerationCfg "https://github.com/harish-nesrekar/vyom/blob/main/generation_config.json"
    click LegacyPkg "https://github.com/harish-nesrekar/vyom/blob/main/backend/package.json"
    click LegacyLock "https://github.com/harish-nesrekar/vyom/blob/main/backend/package-lock.json"
    click LegacyNodeMods "https://github.com/harish-nesrekar/vyom/tree/main/backend/node_modules"

    %% Styles
    classDef frontend fill:#87CEEB,stroke:#000,color:#000
    classDef backend fill:#32CD32,stroke:#000,color:#000
    classDef python fill:#FFA500,stroke:#000,color:#000
    classDef db fill:#A9A9A9,stroke:#000,color:#000
    classDef external fill:#D3D3D3,stroke-dasharray:5 5,color:#000
```
