# vaggiri/ai-website-navigation-assistant architecture

> Architecture diagram of the GitHub repository vaggiri/ai-website-navigation-assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vaggiri/ai-website-navigation-assistant
- Repository: https://github.com/vaggiri/ai-website-navigation-assistant

## Overview

This repository is a **Chrome Extension + dual-backend AI service** for “website navigation assistance”:
- **In-page assistant (content script)** renders a chat UI, captures text/voice queries, inspects the current page DOM/links, highlights the target link, and navigates.
- **Dual-mode AI**:
- **Cloud mode**: Node.js server calls **Google Gemini** for higher-level reasoning/general queries.
- **Local mode**: Python/Flask server hosts a **SentenceTransformer** embedding model for fast/offline semantic matching.
- **Privacy/Firewall**: extension blocks known trackers/third-party domains while allowing sensitive sites to work safely (rulesets / rules.json).

##### Chrome Extension (client-side, inside Chrome)
Key files:
- `manifest.json` — declares extension permissions, content scripts, background, host permissions, possibly rulesets.
- `content.js` — **primary runtime logic** injected into every webpage: UI, hotkeys (`Shift+F`, `Esc`), query handling, link scraping/highlighting/clicking, calling backends.
- `styles.css` — UI styling for the assistant overlay.
- `background.js` — extension background/service worker logic (often used for: network interception, rulesets, message routing, centralized state).
- `popup.html`, `popup.js` — extension popup UI (toggle modes, settings, mic, etc.).
- `newtab.html` — custom new-tab / onboarding or quick entry point.

Also present:
- `debug_repro.js`, `test_gemini.js` — dev/test utilities (don’t put in the main architecture view; optionally in a “Dev/Testing” subview).

##### Node.js “Cloud Mode” server (port 3000)
- `server/server.js` — HTTP API that receives user query/context and calls Gemini.
- `server/package.json`, `package-lock.json` — dependencies, runtime entry.
- `.env` (implied by README) — `GEMINI_API_KEY`.

##### Python “Local Mode” server (port 5000)
- `server/ai_server.py` — Flask API that loads a SentenceTransformer model and performs embedding similarity / link matching.

##### Local training pipeline (offline / dev workflow)
- `server/training_data.json` — supervised examples.
- `server/train.py` — fine-tuning script producing updated local model artifacts (not shown in tree, but implied).

##### User on arbitrary website
- `content.js` renders overlay UI (styled by `styles.css`).
##### User enters query (text or voice)
- Voice: Browser speech APIs or capture in content/popup (depending on implementation) → query text.
##### `content.js` extracts page context
- Collects visible links/buttons/nav items; may capture DOM text snippets.
##### `content.js` decides AI path
- **Local mode** → call `http://localhost:5000/...` (Flask)
- **Cloud mode** → call `http://localhost:3000/...` (Node)
##### Backend returns an answer
- Typically a **best matching link label/URL/selector** + confidence + explanation.
##### Throughout: Firewall/privacy shield blocks trackers
- Implemented via extension rules (ruleset/rules.json) and/or background interception.

This explicitly as:
- **Content Script** → (HTTP) → **Node server** → (HTTPS) → **Gemini**
- **Content Script** → (HTTP) → **Python server** → (local inference) → response
- **Content Script** → DOM highlight/click → **Webpage**
- **Background / Rules Engine** → block/allow network requests → affects Webpage loads and third-party calls

##### Cross-component messaging inside the extension
- `popup.js` ↔ `background.js` (mode toggle/settings stored centrally)
- `popup.js` ↔ `content.js` (open/close assistant, pass configuration)
- `content.js` ↔ `background.js` (optional: for permissions, storage, rules updates, centralized logging)

- **Sidecar UX pattern**: in-page overlay UI (content script) attached to arbitrary websites.
- **Dual-backend strategy** (policy-based routing):
- local fast semantic matching for privacy/offline
- cloud reasoning for general queries
- **Edge privacy enforcement**: request blocking at the extension layer using rulesets (`rules.json`, `_metadata/...`), likely via Chrome’s declarativeNetRequest model.
- **Human-in-the-loop training**: local model fine-tuning using `training_data.json` + `train.py`.

- User
- Chrome Extension
- Visited Website
- Node Cloud Server (3000)
- Python Local Server (5000)
- Google Gemini API
Plus “Tracker/3rd-party domains” as an external entity affected by the firewall.

- Chrome Extension container broken into:
- Content Script (content.js + styles.css)
- Background/Service Worker (background.js)
- Popup UI (popup.html/popup.js)
- Ruleset/Firewall (rules.json + _metadata)
- Node server container:
- REST endpoints
- Gemini client
- Env secrets
- Python server container:
- REST endpoints
- SentenceTransformer model
- Model artifacts
- Training pipeline as a separate container/lane (non-runtime)

`content.js` because it orchestrates:
- UI renderer
- Hotkey handler
- Voice capture → text
- Page link extraction (DOM scraper)
- Mode router (local vs cloud)
- Highlighter + navigator (scroll/click)
- Error handling/fallback heuristics (“change password” fast path)

- **Browser / Extension**
- **User’s Webpage (DOM)**
- **Localhost Services**
- **External Cloud**

This makes it instantly obvious what runs where and what’s remote.

- From `content.js` to backends:
“query + extracted_links/context + mode + (optional) site URL”
- From backends to `content.js`:
“target link/selector + confidence + rationale”
- Firewall:
“blocklist/allowlist rules” applied to “network requests”.

##### Ports and trust boundaries
Add:
- `localhost:3000` Node
- `localhost:5000` Python
- Trust boundary around “External: Gemini API” (keys, privacy).
- That “Cloud mode may transmit page context” (if it does)—mark as a privacy consideration.

Nodes at minimum:
- User
- Chrome (Extension)
- content.js (+ styles.css)
- background.js
- popup.html/popup.js
- rules.json + _metadata rulesets
- Visited Website (DOM)
- Node server (server/server.js, :3000) → Gemini API
- Python server (server/ai_server.py, :5000) → SentenceTransformer model
- Training lane: training_data.json → train.py → model artifacts → ai_server.py
- Third-party trackers (as an external entity affected by firewall)

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =========================
  %% Level 0 — Context (top)
  %% =========================
  subgraph "Level 0 — Context"
    direction TB
    U["User"]:::external
    EXT0["Chrome Extension (Injected Assistant + Privacy Shield)"]:::ext
    WEB0["Visited Website (DOM + Links)"]:::web
    NODE0["Cloud AI Gateway (Node.js) :3000"]:::local
    PY0["Local Embedding Service (Flask) :5000"]:::local
    GEM0["Google Gemini API (External Cloud)"]:::cloud
    TRK0["3rd-party Trackers / Ad Domains"]:::external
  end

  U -->|"uses"| EXT0
  EXT0 -->|"assists_navigation"| WEB0
  EXT0 -->|"HTTP:query+page_context"| NODE0
  EXT0 -->|"HTTP:query+extracted_links"| PY0
  NODE0 -->|"HTTPS:prompt+context"| GEM0
  EXT0 -->|"blocks/filters_requests"| TRK0

  %% =========================
  %% Level 1 — Containers / Swimlanes
  %% =========================
  subgraph "Browser / Extension Runtime (Sidecar UX + Edge Privacy Enforcement)"
    direction TB

    MAN["Manifest (permissions + scripts + host_permissions + rulesets)"]:::ext

    subgraph "Extension UI Surfaces"
      direction TB
      POPHTML["Popup UI (mode toggle/settings)"]:::ext
      POPJS["Popup Controller (state + actions)"]:::ext
      NEWTAB["New Tab / Onboarding"]:::ext
    end

    subgraph "In-page Assistant (Injected into arbitrary sites)"
      direction TB
      CS["Content Script Orchestrator (overlay UI + DOM inspection + navigation)"]:::ext
      CSS["Overlay Styles"]:::ext
    end

    subgraph "Background & Policy"
      direction TB
      BG["Background / Service Worker (central messaging + request handling)"]:::ext
      FW["Network Request Blocker / Tracker Firewall (DNR rules)"]:::security
      META["Ruleset Metadata / Indexed Rulesets (generated artifacts)"]:::security
    end
  end

  subgraph "User's Webpage (Execution Context: DOM)"
    direction TB
    DOM["Page DOM (links/buttons/nav items)"]:::web
    NET["Page Network Requests"]:::web
  end

  subgraph "Localhost Services (Dual-backend Strategy)"
    direction TB

    subgraph "Cloud Mode (localhost:3000)"
      direction TB
      NODE["Node.js HTTP API (Gemini gateway)"]:::local
      NODEENV["Env Secrets (.env: GEMINI_API_KEY)"]:::note
      GEMCLI["Gemini Client"]:::local
      NODEDEPS["Node Runtime Metadata (deps)"]:::note
    end

    subgraph "Local Mode (localhost:5000)"
      direction TB
      PY["Flask HTTP API (Embedding + semantic match)"]:::local
      STM["SentenceTransformer Model (loaded)"]:::local
      ART["Local Model Artifacts"]:::data
    end
  end

  subgraph "External Cloud (Trust Boundary)"
    direction TB
    GEM["Google Gemini API"]:::cloud
    PRIV["Privacy Consideration: cloud_mode_may_transmit_page_context"]:::note
  end

  %% Level 1 wiring
  U -->|"hotkeys/typing/voice"| CS
  CS -->|"renders_overlay"| DOM
  CS -->|"reads:visible_links+context"| DOM
  CS -->|"highlight+scroll+click"| DOM

  CS -->|"HTTP POST:query+extracted_links+site_url"| PY
  PY -->|"inference:embed+similarity"| STM
  STM -->|"uses"| ART
  PY -->|"response:target_selector+confidence+rationale"| CS

  CS -->|"HTTP POST:query+page_context+mode"| NODE
  NODE -->|"reads_key"| NODEENV
  NODE -->|"calls"| GEMCLI
  GEMCLI -->|"HTTPS:prompt+context"| GEM
  GEM -->|"completion/plan"| GEMCLI
  NODE -->|"response:target_label/url/selector+confidence+rationale"| CS

  POPHTML -->|"UI_events"| POPJS
  POPJS -.->|"chrome.runtime.sendMessage (mode/settings)"| BG
  POPJS -.->|"chrome.tabs.sendMessage (open/close assistant)"| CS
  CS -.->|"runtime messaging (state/logging)"| BG

  BG -->|"installs/applies_rulesets"| FW
  FW -->|"uses"| META
  MAN -->|"declares"| CS
  MAN -->|"declares"| BG
  MAN -->|"declares"| POPHTML
  MAN -->|"declares"| NEWTAB
  MAN -->|"declares_ruleset"| FW

  FW -->|"block/allow rules"| NET
  NET -->|"requests_to"| TRK0
  FW -->|"blocks"| TRK0
  DOM -->|"loads_resources_via"| NET

  GEM -->|"within_trust_boundary"| PRIV

  %% =========================
  %% Level 2 — content.js Components
  %% =========================
  subgraph "Level 2 — Content Script Internal Components (content.js)"
    direction TB

    CS_UI["UI Renderer (chat overlay + state)"]:::ext
    CS_KEYS["Hotkey Handler (Shift+F, Esc)"]:::ext
    CS_VOICE["Voice Capture → Text (browser speech APIs)"]:::ext
    CS_DOM["DOM Scraper (links/buttons/nav extraction)"]:::ext
    CS_ROUTE["Mode Router (Local vs Cloud policy)"]:::ext
    CS_CALL["Backend Client (HTTP to :5000 / :3000)"]:::ext
    CS_HL["Highlighter + Navigator (scroll/click/route)"]:::ext
    CS_ERR["Fallback / Error Handling (heuristics)"]:::note
  end

  CS -->|"contains"| CS_UI
  CS -->|"contains"| CS_KEYS
  CS -->|"contains"| CS_VOICE
  CS -->|"contains"| CS_DOM
  CS -->|"contains"| CS_ROUTE
  CS -->|"contains"| CS_CALL
  CS -->|"contains"| CS_HL
  CS -->|"contains"| CS_ERR

  CS_KEYS -->|"open/close"| CS_UI
  CS_VOICE -->|"query_text"| CS_UI
  CS_DOM -->|"extracted_links+context"| CS_ROUTE
  CS_ROUTE -->|"choose_backend"| CS_CALL
  CS_CALL -->|"local_mode"| PY
  CS_CALL -->|"cloud_mode"| NODE
  CS_CALL -->|"response:target+confidence"| CS_HL
  CS_HL -->|"apply_highlight+click"| DOM
  CS_ERR -->|"fallback_on_fail"| CS_HL

  %% =========================
  %% Training / Fine-tuning Lane (offline)
  %% =========================
  subgraph "Offline / Dev Workflow — Human-in-the-loop Training (Non-runtime)"
    direction TB
    TD["Training Data (supervised examples)"]:::data
    TR["Train Script (fine-tune/update)"]:::dev
    UPD["Updated Local Model Artifacts"]:::data
  end

  TD -.->|"input"| TR
  TR -.->|"produces"| UPD
  UPD -.->|"deployed_to"| ART

  %% =========================
  %% Supporting assets (optional)
  %% =========================
  subgraph "Supporting Assets (not core runtime)"
    direction TB
    SITE["Static Landing / Portal Site"]:::support
    SITE_CSS["Portal Styles"]:::support
  end
  SITE -->|"styled_by"| SITE_CSS

  %% =========================
  %% Click events (paths only; not shown in node labels)
  %% =========================
  click MAN "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/manifest.json"
  click CS "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/content.js"
  click CSS "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/styles.css"
  click BG "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/background.js"
  click POPHTML "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/popup.html"
  click POPJS "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/popup.js"
  click NEWTAB "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/newtab.html"
  click FW "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/rules.json"
  click META "https://github.com/vaggiri/ai-website-navigation-assistant/tree/main/_metadata/"
  click META "https://github.com/vaggiri/ai-website-navigation-assistant/tree/main/_metadata/generated_indexed_rulesets/"
  click META "https://github.com/vaggiri/ai-website-navigation-assistant/tree/main/_metadata/generated_indexed_rulesets/_ruleset1/"
  click NODE "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/server/server.js"
  click NODEDEPS "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/server/package.json"
  click NODEDEPS "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/server/package-lock.json"
  click PY "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/server/ai_server.py"
  click TD "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/server/training_data.json"
  click TR "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/server/train.py"
  click SITE "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/website/index.html"
  click SITE_CSS "https://github.com/vaggiri/ai-website-navigation-assistant/blob/main/website/portal.css"

  %% =========================
  %% Styles
  %% =========================
  classDef ext fill:#1f77b4,stroke:#0b3d66,color:#ffffff,stroke-width:1px
  classDef local fill:#2ca02c,stroke:#1b5e20,color:#ffffff,stroke-width:1px
  classDef cloud fill:#ff7f0e,stroke:#8a4b08,color:#ffffff,stroke-width:1px
  classDef web fill:#7f7f7f,stroke:#424242,color:#ffffff,stroke-width:1px
  classDef external fill:#9e9e9e,stroke:#424242,color:#ffffff,stroke-width:1px
  classDef security fill:#6a1b9a,stroke:#3a0a55,color:#ffffff,stroke-width:1px
  classDef data fill:#00897b,stroke:#004d40,color:#ffffff,stroke-width:1px
  classDef note fill:#fff3cd,stroke:#d39e00,color:#333333,stroke-width:1px
  classDef dev fill:#455a64,stroke:#1c313a,color:#ffffff,stroke-width:1px
  classDef support fill:#546e7a,stroke:#263238,color:#ffffff,stroke-width:1px
```
