# sherin-sef-ai/incident-lens-ai architecture

> Architecture diagram of the GitHub repository sherin-sef-ai/incident-lens-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sherin-sef-ai/incident-lens-ai
- Repository: https://github.com/sherin-sef-ai/incident-lens-ai

## Overview

This repo is a **frontend-only, client-side React 19 + TypeScript + Vite** application for **forensic video analysis / accident reconstruction**. The core “engine” is not a local service; it’s **Google Gemini (via `@google/genai`)** accessed from the browser, with **streaming responses + tool use (`googleSearch`)** producing **structured JSON** that drives dashboards/charts and reporting.

- The **browser app** performs **media preprocessing** (frames/audio extraction), orchestration, visualization, and PDF export.
- The **AI inference** and **search grounding** occur via **external Google services**.
- There is **no backend in this repo** (but README explicitly recommends a backend proxy for production).

##### App shell / entry
- `index.html` (Vite HTML entry)
- `index.tsx` (React root mount)
- `App.tsx` (top-level composition, global state orchestration)

##### UI components
- `components/VideoUploader.tsx`
- evidence ingestion (drag/drop, file selection)
- likely emits “video selected + metadata” events into app state
- `components/AnalysisDashboard.tsx`
- main “case workspace”: timeline, stream view, knobs/sandbox, chat-like Q&A, etc.
- `components/AnalysisCharts.tsx`
- charts (Recharts radar/pie, etc.) and any computed views from structured JSON

##### Service layer (important boundary)
- `services/geminiService.ts`
- all calls to Gemini API (streaming `generateContentStream`)
- tool use / grounding via `googleSearch`
- prompt building + strict JSON response handling (typed structures)

##### Shared contract/config
- `types.ts` (typed JSON schema the model must output; drives UI)
- `constants.ts` (model name, prompt templates, chart constants, etc.)
- `metadata.json` (training scenarios or sample dataset metadata; represent as “local bundled scenarios/data”)
- `package.json`, `vite.config.ts`, `tsconfig.json` (build/runtime boundaries; show in deployment view)

**Goal:** who uses it, what external systems it depends on.

- **User (Claims/Legal/Fleet)**
- **Incident Lens AI Web App (Browser SPA)**
- **Google Gemini API (Gemini 3 Pro)**
- **Google Search Tool / Grounding** (as part of Gemini tool-use; still show explicitly)
- (Optional) **Static Hosting/CDN** (where the Vite build is deployed)

- User ↔ Web App: upload video, review dashboard, export PDF
- Web App → Gemini API: frames+audio+prompt (multimodal)
- Gemini API ↔ Google Search: tool calls for statutes/case law grounding
- Web App ← Gemini API: streaming tokens / structured JSON + citations

**Goal:** what runs where.

Containers (be explicit that this repo only contains one runtime container):
- **Browser SPA (React/Vite)**
Subsystems inside it:
- UI layer (components)
- Media preprocessing (Frame extraction, AudioContext waveform extraction)
- AI Orchestrator (Gemini service)
- Visualization (Recharts, Canvas annotations)
- Reporting (jsPDF client-side)
- **External: Gemini Service**
- **External: Search Grounding**

- “Client-side API key present (demo)” boundary
- “Production: backend proxy recommended” note

**Goal:** show internal separation inside the SPA.

- `App.tsx` (state + routing between uploader/dashboard)
- `VideoUploader.tsx` → emits `EvidenceAsset` / selected scenario
- “Media Processing Module” (even if not a separate file, represent as a logical component used by App/Dashboard)
- Frame sampler
- Audio extractor (AudioContext)
- `geminiService.ts`
- request builder (prompt + attachments)
- stream handler (incremental updates)
- JSON validator/parser (maps to `types.ts`)
- `AnalysisDashboard.tsx`
- live analysis stream viewer
- timeline controller
- variable sandbox inputs (friction, etc.)
- `AnalysisCharts.tsx`
- renders `Recharts` radar/pie etc. from typed JSON
- “PDF Exporter” (logical component; implemented somewhere in App/Dashboard)
- `types.ts` as “Shared Schema/Contracts”
- `constants.ts` as “Prompts/Model Config”
- `metadata.json` as “Bundled Scenario Metadata”

- Uploader → App state: `videoFile`, `scenarioId`, `caseMetadata`
- App/Dashboard → Media Processing: `videoFile` → `{frames[], audioWaveform, timestamps}`
- App/Dashboard → `geminiService`: `{frames, audio, promptConfig}` → streaming
- `geminiService` → App/Dashboard: `{streamEvents, partialFindings, finalStructuredJSON}`
- Dashboard → Charts: `analysisResultJSON`
- Dashboard → PDF Exporter: `analysisResultJSON + citations + screenshots`

**Goal:** capture the “reasoning stream” lifecycle precisely.

##### App triggers preprocessing
- extract frames (sampling rate)
- extract audio waveform (WAV/PCM representation)
##### Gemini streams back
- partial text/thinking trace (if shown)
- tool-use calls: `googleSearch(query)`
- returns grounded citations
- emits structured JSON blocks (final)
##### User exports PDF (jsPDF) containing
- findings
- charts
- citations / chain-of-custody notes (as supported)

**Goal:** show build artifacts and runtime environment.

Include:
- Developer machine: Vite dev server
- CI/build step: `vite build` → static assets
- Hosting: static site host/CDN
- Runtime: user browser
- External: Gemini endpoints

Call out env handling:
- `.env API_KEY` injected at build/runtime (client-side in this demo)

##### AI inference is remote (Gemini API); the app is primarily an orchestrator + viewer.
4) **Tool grounding is remote** (Gemini calling search) and should be drawn as a separate external dependency.
5) **UI is driven by typed structured JSON** (critical: `types.ts` is the contract between AI output and UI).
6) **Streaming** is not a simple request/response; it’s incremental updates that drive “Live Analysis Stream”.

##### Architectural patterns/design principles evident here

- **SPA + Service Layer boundary:** React components call a dedicated `geminiService.ts` (good separation).
- **Schema-first UI:** `types.ts` defines the output contract; UI components assume typed data.
- **Streaming/event-driven UI updates:** analysis is shown progressively; dashboard re-renders on stream events.
- **Client-side pipeline orchestration:** preprocessing → inference → render → export all in-browser.

##### “Don’t overthink it” — maximize component separation
- Evidence ingestion
- Media preprocessing
- AI orchestration/service
- Stream handling
- Schema validation
- Visualization (charts/timeline)
- Reporting/export

##### Optional: Production-target extension (annotate as “future architecture”)
- Backend API (proxy Gemini calls, protect keys)
- Object storage for video evidence (S3/GCS)
- Database for cases/metadata
- AuthN/AuthZ (enterprise users)
- Audit log / chain-of-custody immutability

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =========================================================
  %% Level 1: System Context (who uses it / external deps)
  %% =========================================================
  subgraph "System Context (Level 1)"
    direction TB

    User["User (Claims/Legal/Fleet)"]:::external

    subgraph "Incident Lens AI Web App (Browser SPA)"
      direction TB
      WebApp["React 19 + TypeScript SPA"]:::spa
    end

    CDN["Static Hosting / CDN (Vite build)"]:::infra
    Gemini["Google Gemini API (Gemini 3 Pro)"]:::externalSvc
    Search["Google Search Tool / Grounding (tool-use)"]:::externalSvc

    User <-->|"uploadVideo/reviewDashboard/exportPDF"| WebApp
    CDN -->|"serveStaticAssets"| WebApp
    WebApp -->|"multimodalRequest(frames+audio+prompt)"| Gemini
    Gemini <-->|"toolCalls(googleSearch)"| Search
    Gemini -->|"streamingEvents(tokens+citations+structuredJSON)"| WebApp
  end

  %% =========================================================
  %% Level 2/3: Browser Container + Components (file-mapped)
  %% =========================================================
  subgraph "Browser Runtime / Trust Boundary (Client-side only; no backend in repo)"
    direction TB

    NoteKey["Note: API key in client (demo);\nbackend proxy recommended for production"]:::noteDanger

    %% --- App shell / entry ---
    subgraph "App Shell / Entry"
      direction TB
      IndexHtml["index.html"]:::build
      IndexTsx["index.tsx"]:::build
      App["App.tsx"]:::ui
    end

    IndexHtml -->|"loadsBundle"| IndexTsx
    IndexTsx -->|"mountReactRoot"| App

    %% --- UI Layer ---
    subgraph "UI Layer (React Components)"
      direction TB
      Uploader["components/VideoUploader.tsx"]:::ui
      Dashboard["components/AnalysisDashboard.tsx"]:::ui
      Charts["components/AnalysisCharts.tsx"]:::ui
      PDF["PDF Exporter (jsPDF client-side)"]:::report
    end

    App -->|"compose/render"| Uploader
    App -->|"compose/render"| Dashboard
    Dashboard -->|"rendersFromTypedJSON"| Charts
    Dashboard -->|"exportReport"| PDF

    %% --- Local Media Preprocessing Pipeline ---
    subgraph "Local Media Preprocessing (In-Browser)"
      direction TB
      MediaProc["Media Processing Module"]:::localProc
      Frames["Frame Sampler"]:::localProc
      Audio["Audio Extractor (AudioContext)"]:::localProc
      MediaProc -->|"extractFrames"| Frames
      MediaProc -->|"extractAudioWaveform"| Audio
    end

    Uploader -->|"VideoFile(MP4/MOV)+metadata"| App
    App -->|"startPreprocess(videoFile)"| MediaProc
    Frames -->|"SampledFrames(JPEG/PNG)+timestamps"| Dashboard
    Audio -->|"AudioWaveform(PCM/WAV)"| Dashboard

    %% --- Service Layer Boundary ---
    subgraph "Service Layer (Integration)"
      direction TB
      GeminiSvc["services/geminiService.ts"]:::service
      ReqBuild["Request Builder (prompt+attachments)"]:::service
      StreamH["Stream Handler (incremental updates)"]:::service
      JsonParse["JSON Validator/Parser"]:::service
      GeminiSvc -->|"buildRequest"| ReqBuild
      GeminiSvc -->|"handleStream"| StreamH
      GeminiSvc -->|"parseValidate"| JsonParse
    end

    Dashboard -->|"generateContentStream(frames+audio+promptConfig)"| GeminiSvc
    GeminiSvc -->|"streamEvents(partial+final)"| Dashboard

    %% --- Shared contracts/config/data ---
    subgraph "Shared Contract / Config / Local Data"
      direction TB
      Types["types.ts (Typed JSON schema/contract)"]:::shared
      Consts["constants.ts (Model+Prompts+Chart constants)"]:::shared
      Meta["metadata.json (Bundled scenarios/metadata)"]:::data
    end

    App -.->|"imports"| Types
    Dashboard -.->|"imports"| Types
    Charts -.->|"imports"| Types
    GeminiSvc -.->|"imports"| Types

    App -.->|"imports"| Consts
    Dashboard -.->|"imports"| Consts
    GeminiSvc -.->|"imports"| Consts
    Uploader -.->|"readsScenarios"| Meta

    JsonParse -->|"validateAgainstSchema"| Types

    %% --- External calls from browser ---
    GeminiSvc -->|"HTTPS: Gemini API\n(streaming)"| Gemini
    Gemini -->|"citations+groundedResults"| Dashboard
    Gemini <-->|"toolCalls(googleSearch)"| Search
  end

  %% =========================================================
  %% Level 4: Lifecycle (Sequence summary as flow)
  %% =========================================================
  subgraph "Analysis Lifecycle (Streaming, event-driven UI updates)"
    direction TB
    Step1["1) Upload Evidence"]:::step
    Step2["2) Preprocess (frames + audio)"]:::step
    Step3["3) Stream AI Analysis (Gemini)"]:::step
    Step4["4) Tool Grounding (Search)"]:::step
    Step5["5) Validate Structured JSON (types.ts)"]:::step
    Step6["6) Live Dashboard + Charts Update"]:::step
    Step7["7) Export PDF (jsPDF)"]:::step

    Step1 --> Step2 --> Step3 --> Step4 --> Step5 --> Step6 --> Step7
  end

  Uploader -->|"userAction"| Step1
  MediaProc -->|"outputs"| Step2
  GeminiSvc -->|"generateContentStream"| Step3
  Search -->|"grounding"| Step4
  Types -->|"schema"| Step5
  Dashboard -->|"renders"| Step6
  PDF -->|"createsReport"| Step7

  %% =========================================================
  %% Deployment View (current repo reality)
  %% =========================================================
  subgraph "Deployment (Current Repo)"
    direction TB
    Dev["Developer Machine"]:::infra
    ViteDev["Vite Dev Server"]:::infra
    Build["Build Step: vite build"]:::infra
    Assets["Static Assets (HTML/CSS/JS)"]:::infra
    Host["Static Host / CDN"]:::infra
    Browser["User Browser Runtime"]:::spa

    Dev -->|"npm run dev"| ViteDev
    Dev -->|"npm run build"| Build
    Build -->|"produces"| Assets
    Assets -->|"deploy"| Host
    Host -->|"serves"| Browser
    Browser -->|"calls"| Gemini
  end

  %% Tooling boundary files
  subgraph "Tooling / Config (Repo Boundaries)"
    direction TB
    Pkg["package.json"]:::build
    ViteCfg["vite.config.ts"]:::build
    TsCfg["tsconfig.json"]:::build
  end

  Pkg -.->|"definesScriptsDeps"| ViteDev
  Pkg -.->|"definesBuild"| Build
  ViteCfg -.->|"configuresBundling"| Build
  TsCfg -.->|"typeChecking"| Build

  %% =========================================================
  %% Optional: Production-target extension (annotated as future)
  %% =========================================================
  subgraph "Production Extension (Future Architecture - not in repo)"
    direction TB

    ProdNote["Future: Protect keys + persist cases + audit trail"]:::note

    subgraph "Backend (Recommended Proxy)"
      direction TB
      Proxy["Backend API Proxy"]:::backend
      Auth["AuthN/AuthZ"]:::backend
      Audit["Audit Log / Chain-of-Custody"]:::backend
      DB["Case Database"]:::db
      Store["Object Storage (Video Evidence)"]:::db
      Auth --> Proxy
      Proxy --> Audit
      Proxy --> DB
      Proxy --> Store
    end

    Browser2["Browser SPA"]:::spa
    Browser2 -->|"uploadEvidence"| Store
    Browser2 -->|"auth"| Auth
    Browser2 -->|"AIRequests(noClientKey)"| Proxy
    Proxy -->|"Gemini API"| Gemini
    Proxy <-->|"toolCalls"| Search
  end

  %% =========================================================
  %% Click events (from component_mapping)
  %% =========================================================
  click IndexHtml "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/index.html"
  click IndexTsx "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/index.tsx"
  click App "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/App.tsx"
  click Uploader "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/components/VideoUploader.tsx"
  click Dashboard "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/components/AnalysisDashboard.tsx"
  click Charts "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/components/AnalysisCharts.tsx"
  click GeminiSvc "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/services/geminiService.ts"
  click Types "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/types.ts"
  click Consts "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/constants.ts"
  click Meta "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/metadata.json"
  click Pkg "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/package.json"
  click ViteCfg "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/vite.config.ts"
  click TsCfg "https://github.com/sherin-sef-ai/incident-lens-ai/blob/main/tsconfig.json"

  %% =========================================================
  %% Styles
  %% =========================================================
  classDef spa fill:#E6F0FF,stroke:#2563EB,stroke-width:2,color:#0B1F44
  classDef ui fill:#DBEAFE,stroke:#1D4ED8,stroke-width:1.5,color:#0B1F44
  classDef service fill:#F3E8FF,stroke:#7C3AED,stroke-width:1.5,color:#2E1065
  classDef localProc fill:#FFEDD5,stroke:#F97316,stroke-width:1.5,color:#431407
  classDef report fill:#E0F2FE,stroke:#0284C7,stroke-width:1.5,color:#082F49
  classDef shared fill:#F1F5F9,stroke:#334155,stroke-width:1.2,color:#0F172A
  classDef data fill:#ECFDF5,stroke:#10B981,stroke-width:1.5,color:#064E3B
  classDef external fill:#FFFFFF,stroke:#475569,stroke-width:1.5,color:#0F172A
  classDef externalSvc fill:#DCFCE7,stroke:#16A34A,stroke-width:2,color:#052E16
  classDef infra fill:#F8FAFC,stroke:#64748B,stroke-width:1.2,color:#0F172A
  classDef build fill:#EEF2FF,stroke:#4F46E5,stroke-width:1.2,color:#1E1B4B
  classDef backend fill:#FFE4E6,stroke:#E11D48,stroke-width:1.5,color:#4C0519
  classDef db fill:#FFF7ED,stroke:#C2410C,stroke-width:1.5,color:#431407
  classDef noteDanger fill:#FEE2E2,stroke:#DC2626,stroke-width:2,color:#450A0A
  classDef note fill:#FEF9C3,stroke:#CA8A04,stroke-width:1.5,color:#422006
  classDef step fill:#FFFFFF,stroke:#0EA5E9,stroke-width:1.2,color:#082F49,stroke-dasharray: 3 3
```
