# harshaltupe12/screenshare architecture

> Architecture diagram of the GitHub repository harshaltupe12/screenshare, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harshaltupe12/screenshare
- Repository: https://github.com/harshaltupe12/screenshare

## Overview

##### Project Type and Purpose
- This is a full-stack, real-time web application (“Jerry – AI-Powered Screen Assistant”) that lets users share their browser screen, runs OCR on the frames, and returns contextual AI responses (text and voice) via WebSockets.
- Frontend: Next.js 14 + TypeScript, React components, MediaDevices API, canvas capture, react-speech-kit.
- Backend: Node.js + Express + Socket.IO, MongoDB, JWT auth, Tesseract OCR, Gemini/Hugging Face AI integrations.

##### File Structure Analysis
- Top level: `backend/` and `frontend/`.
- `backend/src/`: MVC-style split into controllers, routes, services, models, middleware, utils, plus config files and test scripts.
- `frontend/app/` (Next.js App Router pages), `frontend/components/` (UI and domain components), `frontend/lib/` (API, auth, WebSocket helpers), plus config (tsconfig, ESLint, postcss, next.config).

##### Key README Insights
- Real-time features: WebSocket for screen frames and AI responses.
- Authentication: JWT with refresh, password hashing, account lockout.
- AI pipeline: screen capture → base64 → WebSocket → OCR → AI API → response → persistence → WebSocket back.
- Data storage: MongoDB Atlas, four main collections (User, Meeting, Message, AISession).

##### Diagramming the System
a. Main Components
- Frontend Browser (Next.js App)
- Backend API Server (Express + Socket.IO)
- Database (MongoDB Atlas)
- OCR Engine (Tesseract via `eng.traineddata`)
- AI Providers (Gemini AI, Hugging Face)
- Auth subsystem (JWT, bcrypt)
- External Services: WebSocket clients, TTS/ASR in browser

b. Relationships & Data Flows
1. User → Frontend UI (React pages/components)
2. Frontend REST → Backend Auth routes → issue JWT
3. User triggers “Share Screen” → MediaDevices API + Canvas → base64 images
4. Frontend WebSocket → Backend `/ai-session` route
5. Backend Service chain:
- OCR Service extracts text
- AI Service calls external AI provider
- Persist to MongoDB (models: AISession, Message)
6. Backend WebSocket → Frontend → display chat + play TTS
7. Frontend Speech Recognition → text input → loop

c. Architectural Patterns & Principles
- Separation of concerns (MVC on backend, feature-based directories on frontend).
- Event-driven, real-time communication (Socket.IO).
- Stateless JWT authentication with middleware guards.
- Service layer abstracts AI, OCR, TTS.
- Environment-based configuration (`aiConfig.js`, `database.js`).

d. Key Technologies/Frameworks
- Frontend: Next.js 14, React, TypeScript, react-speech-kit, WebSocket API, MediaDevices
- Backend: Node.js, Express, Socket.IO, Mongoose, Tesseract, Axios (for AI APIs), bcrypt, jsonwebtoken
- Database: MongoDB Atlas
- Deployment: environment variables, cross-origin setup (`FRONTEND_URL`)

##### Essential Details to Capture
- All four backend controllers and their corresponding routes (auth, ai, aiSession, meetings).
- Service dependencies: aiController → aiService → external AI; aiSessionController → ocrService → aiService → ttsService.
- JWT flow: client sends token → auth middleware validates → requests processed.
- WebSocket handshake path (`/socket.io` or custom namespace).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client (Browser)"
        direction TB
        UI["Next.js UI Pages"]:::frontend
        ScreenShare["Screen Share Module"]:::frontend
        AIChat["AI Chat Component"]:::frontend
        MeetingControls["Meeting Controls"]:::frontend
        Navbar["Navbar/Layout"]:::frontend
        AuthPages["Auth Pages"]:::frontend
        Dashboard["Dashboard Page"]:::frontend
        MeetingRoom["Meeting Room Page"]:::frontend
        Profile["Profile Page"]:::frontend
        RestApi["REST API Helpers"]:::frontend
        AuthLib["Auth Helpers (JWT)"]:::frontend
        WSClient["WebSocket Client"]:::frontend
        SpeechHooks["Speech Hooks (TTS/ASR)"]:::frontend
    end

    subgraph "Server (Cloud)"
        direction TB
        Entry["Server Entry Point"]:::backend
        subgraph "Config"
            AiConfig["AI Config"]:::backend
            DbConfig["Database Config"]:::backend
        end
        subgraph "Controllers"
            AuthController["Auth Controller"]:::backend
            AiController["AI Controller"]:::backend
            AiSessionController["AI Session Controller"]:::backend
            MeetingController["Meeting Controller"]:::backend
        end
        subgraph "Routes"
            AuthRoutes["Auth Routes"]:::backend
            AiRoutes["AI Routes"]:::backend
            AiSessionRoutes["AI Session Routes"]:::backend
            MeetingRoutes["Meeting Routes"]:::backend
        end
        subgraph "Services"
            OcrService["OCR Service"]:::backend
            AiService["AI Service"]:::backend
            AiSessionService["AI Session Service"]:::backend
            TtsService["TTS Service"]:::backend
        end
        subgraph "Middleware & Utils"
            JwtUtils["JWT Utils"]:::backend
            AuthMiddleware["Auth Middleware"]:::backend
        end
        subgraph "Models"
            UserModel["User Model"]:::db
            MeetingModel["Meeting Model"]:::db
            MessageModel["Message Model"]:::db
            AISessionModel["AISession Model"]:::db
        end
        subgraph "Tests & Docs"
            Tests["Test Scripts"]:::backend
            MongoDoc["MongoDB Setup Doc"]:::backend
            AIDoc["AI Engine Setup Doc"]:::backend
        end
    end

    CloudDb[(MongoDB Atlas)]:::db
    extGemini(("Gemini AI")):::external
    extHF(("Hugging Face")):::external

    UI --> ScreenShare
    UI --> AIChat
    UI --> MeetingControls
    UI --> Navbar
    AuthPages --> RestApi
    RestApi --> AuthRoutes
    AuthRoutes --> AuthController
    AuthController --> JwtUtils
    AuthController --> UserModel
    AuthController --> DbConfig
    AuthController --> Entry

    MeetingControls --> RestApi
    Dashboard --> RestApi
    Profile --> RestApi

    RestApi --> AuthLib
    RestApi --> Entry

    ScreenShare -->|"canvas→base64"| WSClient
    MeetingRoom --> WSClient

    WSClient -.->|"socket.io connect"| Entry
    Entry --> AiSessionController
    AiSessionController --> AuthMiddleware
    AiSessionController --> OcrService
    OcrService -->|"image→text"| AiService
    AiService -->|"API call"| extGemini
    AiService -->|"API call"| extHF
    AiService --> AiSessionService
    AiSessionService --> TtsService
    AiSessionService --> MessageModel
    AiSessionService --> AISessionModel
    AiSessionController --> WSClient

    Entry --> AuthRoutes
    Entry --> AiRoutes
    Entry --> AiSessionRoutes
    Entry --> MeetingRoutes

    AiRoutes --> AiController
    AiController --> AiService

    MeetingRoutes --> MeetingController
    MeetingController --> MeetingModel

    AuthRoutes --> AuthMiddleware

    AuthController -- Saved Users --> UserModel
    MeetingController -- Saved Meetings --> MeetingModel

    Entry --> AiConfig
    Entry --> DbConfig

    UserModel --> CloudDb
    MeetingModel --> CloudDb
    MessageModel --> CloudDb
    AISessionModel --> CloudDb

    Tests -.-> Entry
    MongoDoc -.-> DbConfig
    AIDoc -.-> AiConfig

    classDef frontend fill:#D0E8FF,stroke:#0366d6,color:#0366d6;
    classDef backend fill:#E6F4EA,stroke:#22863A,color:#22863A;
    classDef db fill:#FFEFD5,stroke:#DB8E00,color:#DB8E00;
    classDef external fill:#F5E1FF,stroke:#6F42C1,color:#6F42C1;

    click UI "https://github.com/harshaltupe12/screenshare/tree/master/frontend/app"
    click ScreenShare "https://github.com/harshaltupe12/screenshare/blob/master/frontend/components/meeting/ScreenShare.tsx"
    click AIChat "https://github.com/harshaltupe12/screenshare/blob/master/frontend/components/meeting/AIChat.tsx"
    click MeetingControls "https://github.com/harshaltupe12/screenshare/blob/master/frontend/components/meeting/MeetingControls.tsx"
    click Navbar "https://github.com/harshaltupe12/screenshare/blob/master/frontend/components/layout/ConditionalNavbar.tsx"
    click RestApi "https://github.com/harshaltupe12/screenshare/blob/master/frontend/lib/api.ts"
    click AuthLib "https://github.com/harshaltupe12/screenshare/blob/master/frontend/lib/auth.ts"
    click WSClient "https://github.com/harshaltupe12/screenshare/blob/master/frontend/lib/websocket.ts"
    click SpeechHooks "https://github.com/harshaltupe12/screenshare/blob/master/frontend/react-speech-kit.d.ts"
    click AuthPages "https://github.com/harshaltupe12/screenshare/blob/master/frontend/app/auth/page.tsx"
    click Dashboard "https://github.com/harshaltupe12/screenshare/blob/master/frontend/app/dashboard/page.tsx"
    click MeetingRoom "https://github.com/harshaltupe12/screenshare/blob/master/frontend/app/meeting-room/[meetingId]/page.tsx"
    click Profile "https://github.com/harshaltupe12/screenshare/blob/master/frontend/app/profile/page.tsx"

    click Entry "https://github.com/harshaltupe12/screenshare/blob/master/backend/index.js"
    click AiConfig "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/config/aiConfig.js"
    click DbConfig "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/config/database.js"
    click AuthController "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/controllers/authController.js"
    click AuthRoutes "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/routes/auth.js"
    click JwtUtils "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/utils/jwt.js"
    click AuthMiddleware "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/middleware/auth.js"
    click AiController "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/controllers/aiController.js"
    click AiRoutes "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/routes/ai.js"
    click AiSessionController "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/controllers/aiSessionController.js"
    click AiSessionRoutes "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/routes/aiSession.js"
    click MeetingController "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/controllers/meetingController.js"
    click MeetingRoutes "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/routes/meetings.js"
    click OcrService "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/services/ocrService.js"
    click AiService "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/services/aiService.js"
    click AiSessionService "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/services/aiSessionService.js"
    click TtsService "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/services/ttsService.js"
    click UserModel "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/models/User.js"
    click MeetingModel "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/models/Meeting.js"
    click MessageModel "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/models/Message.js"
    click AISessionModel "https://github.com/harshaltupe12/screenshare/blob/master/backend/src/models/AISession.js"
    click Tests "https://github.com/harshaltupe12/screenshare/blob/master/backend/test-context-aware.js"
    click MongoDoc "https://github.com/harshaltupe12/screenshare/blob/master/backend/MONGODB_SETUP.md"
    click AIDoc "https://github.com/harshaltupe12/screenshare/blob/master/backend/AI_SETUP.md"
```
