How tanish-8/nexvision works
NexVision is a privacy-first browser-agent prototype. A user submits a task through the extension; the background runtime perceives the page, sanitizes sensitive information locally, grounds the task to page elements, plans a validated action, and executes it in the browser before continuing its bounded loop. The README documents this end-to-end flow and the DOM fallback; source excerpts show the service-worker dispatch, perception/privacy calls, and planner/action validation. Some implementation details are unsampled, so those links are shown only where documented.
Main entry points: orchestrator.ts Perception orchestrator · demoRunner.ts Bounded agent loop · service-worker.ts Service worker · messaging.ts Extension messaging
Components 21
Browser extension
- Extension popup
extension/src/popup/popup.ts - Service worker
extension/src/background/service-worker.ts - Extension messaging
extension/src/shared/messaging.ts - DOM executor
extension/src/content/domExecutor.ts
Page perception
- DOM perception
extension/src/content/domPerception.ts - Screenshot capture
extension/src/background/screenshot.ts - Perception orchestrator
extension/src/background/orchestrator.ts - Vision adapter
extension/src/background/llamaVisionAdapter.ts - Local vision runtime
vision/src/llamaInference.ts - Unified page state
extension/src/shared/types.ts
Local privacy
- Privacy detector
extension/src/privacy/detector.ts - Page sanitizer
extension/src/privacy/sanitizer.ts
Planning and execution
- Element grounding
extension/src/shared/grounding.ts - Local agent
extension/src/background/localAgent.ts - Action planner
extension/src/shared/planner.ts - Action validation
extension/src/shared/actions.ts - Bounded agent loop
extension/src/background/demoRunner.ts - Action executor
extension/src/background/executor.ts
Other components
- User task
- Browser page
- Local Qwen server
Connections 26
- User task → Extension popup: submits task
README.md - Extension popup → Service worker: sends request
README.md - Service worker → Extension messaging: dispatches messages
extension/src/background/service-worker.ts - Service worker → Bounded agent loop: starts agent run
extension/src/background/service-worker.ts - Browser page → DOM perception: provides DOM
README.md - DOM perception → Unified page state: builds page state
README.md - Service worker → Screenshot capture: captures screenshot
extension/src/background/service-worker.ts - Screenshot capture → Perception orchestrator: supplies image
README.md - Perception orchestrator → Vision adapter: requests perception
extension/src/background/orchestrator.ts - Vision adapter → Local vision runtime: uses inference
- Local vision runtime → Local Qwen server: sends inference request
vision/src/llamaInference.ts - Perception orchestrator → Unified page state: unifies page state
README.md - Perception orchestrator → Privacy detector: checks findings
extension/src/background/orchestrator.ts - Privacy detector → Page sanitizer: provides findings
extension/src/privacy/sanitizer.ts - Unified page state → Page sanitizer: sanitizes page state
README.md - Page sanitizer → Element grounding: provides safe state
README.md - Element grounding → Action planner: grounds task
README.md - Local agent → Action planner: requests plan
extension/src/background/localAgent.ts - Action planner → Action validation: creates and validates
extension/src/shared/planner.ts - Bounded agent loop → Local agent: decomposes task
extension/src/background/demoRunner.ts - Bounded agent loop → Perception orchestrator: runs perception
extension/src/background/demoRunner.ts - Bounded agent loop → Action executor: executes action
README.md - Action executor → Extension messaging: uses tab messaging
extension/src/background/executor.ts - Extension messaging → DOM executor: routes browser action
- DOM executor → Browser page: changes page
README.md - Browser page → Bounded agent loop: is re-perceived
README.md