# shuishen-cloud/blinderdetector architecture

> Architecture diagram of the GitHub repository shuishen-cloud/blinderdetector, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shuishen-cloud/blinderdetector
- Repository: https://github.com/shuishen-cloud/blinderdetector

## Overview

灵眸伴途 is a visual-language-model travel assistant for people with visual impairments. Its principal workflow sends camera frames to backend perception, safety, navigation, and emergency logic; eligible announcements pass a gate and reach the user over a WebSocket for display, speech, and haptics. The frontend also supports destination voice input and route visualization, while a separate sender page simulates frame capture. The README documents optional cloud, map-routing, detector, and ASR integrations; several subsystem details are based on documentation rather than sampled implementation.

Main entry points: [`app/api/envelope.py`](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/envelope.py) (Announcement envelope), [`web/js/net.js`](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/net.js) (Client transport), [`web/js/main.js`](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/main.js) (App assembly), [`app/main.py`](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/main.py) (Application setup)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Frame capture"]
  node_sender["Frame sender<br/>[sender.js]"]
end

subgraph group_backend["Backend transport"]
  node_app["Application setup<br/>[main.py]"]
  node_routes["HTTP and WS routes<br/>[routes.py]"]
  node_uploads["Frame upload<br/>[uploads.py]"]
  node_asr_api["ASR endpoint<br/>[speech.py]"]
  node_envelope["Announcement envelope<br/>[envelope.py]"]
  node_hub["Broadcast hub<br/>[hub.py]"]
  node_arbiter["Announcement gate<br/>[arbiter.py]"]
  node_contracts["Frame contracts<br/>[contracts.py]"]
  node_config["Runtime configuration<br/>[config.py]"]
end

subgraph group_assist["Assistance logic"]
  node_perception["Scene perception<br/>[perception.py]"]
  node_safety["Safety analysis<br/>[safety.py]"]
  node_navigation["Route planning<br/>[navigation.py]"]
  node_emergency["Emergency states<br/>[emergency.py]"]
  node_fall["Fall detection<br/>[fall.py]"]
  node_sos["SOS escalation<br/>[sos.py]"]
  node_phrasing["Announcement phrasing<br/>[phrasing.py]"]
end

subgraph group_integrations["Providers and routing"]
  node_registry["Implementation registry<br/>[registry.py]"]
  node_providers["Vision providers<br/>[openai_compat.py]"]
  node_detector["Object detector<br/>[qwen_vl.py]"]
  node_router["Map router<br/>[baidu.py]"]
  node_asr["Speech recognizer<br/>[dashscope.py]"]
end

subgraph group_frontend["User experience"]
  node_net["Client transport<br/>[net.js]"]
  node_main["App assembly<br/>[main.js]"]
  node_ui["Announcement UI<br/>[ui.js]"]
  node_speech["Speech and haptics<br/>[speech.js]"]
  node_navigation_ui["Destination input<br/>[nav.js]"]
  node_voice["Voice input<br/>[voice.js]"]
  node_map["Route display<br/>[map.js]"]
end

node_user(("User"))
node_browser(("Browser client"))

node_user -->|"provides frames"| node_sender
node_sender -->|"uploads frames"| node_net
node_net -->|"POSTs frames"| node_uploads
node_uploads -->|"builds response"| node_envelope
node_app -->|"builds routes"| node_routes
node_app -->|"loads implementations"| node_registry
node_app -->|"creates hub"| node_hub
node_app -->|"creates envelope"| node_envelope
node_app -->|"creates gate"| node_arbiter
node_routes -->|"routes uploads"| node_uploads
node_routes -->|"routes audio"| node_asr_api
node_routes -->|"routes requests"| node_envelope
node_routes -->|"serves stream"| node_hub
node_envelope -->|"filters announcements"| node_arbiter
node_envelope -->|"broadcasts announcements"| node_hub
node_envelope -->|"uses Frame"| node_contracts
node_envelope -.->|"invokes layer"| node_perception
node_envelope -.->|"invokes layer"| node_safety
node_envelope -.->|"invokes layer"| node_navigation
node_envelope -.->|"invokes layer"| node_emergency
node_perception -.->|"uses vision model"| node_providers
node_safety -.->|"detects obstacles"| node_detector
node_safety -->|"phrases warnings"| node_phrasing
node_navigation -.->|"requests routes"| node_router
node_navigation -.->|"phrases guidance"| node_phrasing
node_emergency -.->|"tracks falls"| node_fall
node_emergency -.->|"advances SOS"| node_sos
node_asr_api -->|"selects recognizer"| node_registry
node_asr_api -.->|"transcribes audio"| node_asr
node_config -->|"selects implementations"| node_registry
node_browser -.->|"uses client transport"| node_net
node_net -->|"receives WS stream"| node_hub
node_main -->|"connects stream"| node_net
node_main -->|"initializes UI"| node_ui
node_main -->|"initializes navigation"| node_navigation_ui
node_main -->|"initializes voice"| node_voice
node_main -->|"initializes map"| node_map
node_net -->|"delivers announcements"| node_ui
node_ui -->|"speaks announcements"| node_speech
node_voice -->|"uploads audio"| node_net
node_voice -->|"submits destination"| node_navigation_ui
node_navigation_ui -->|"requests route"| node_net
node_main -->|"forwards announcements"| node_map

click node_sender "https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/sender.js"
click node_net "https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/net.js"
click node_main "https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/main.js"
click node_ui "https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/ui.js"
click node_speech "https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/speech.js"
click node_navigation_ui "https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/nav.js"
click node_voice "https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/voice.js"
click node_map "https://github.com/shuishen-cloud/blinderdetector/blob/main/web/map.js"
click node_app "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/main.py"
click node_routes "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/routes.py"
click node_uploads "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/uploads.py"
click node_asr_api "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/speech.py"
click node_envelope "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/envelope.py"
click node_hub "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/hub.py"
click node_arbiter "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/arbiter.py"
click node_contracts "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/contracts.py"
click node_perception "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/layers/perception.py"
click node_safety "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/layers/safety.py"
click node_navigation "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/layers/navigation.py"
click node_emergency "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/layers/emergency.py"
click node_fall "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/rules/fall.py"
click node_sos "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/rules/sos.py"
click node_phrasing "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/rules/phrasing.py"
click node_registry "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/registry.py"
click node_providers "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/providers/openai_compat.py"
click node_detector "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/detectors/qwen_vl.py"
click node_router "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/routers/baidu.py"
click node_asr "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/asr/dashscope.py"
click node_config "https://github.com/shuishen-cloud/blinderdetector/blob/main/app/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sender,node_user,node_browser toneBlue
class node_app,node_routes,node_uploads,node_asr_api,node_envelope,node_hub,node_arbiter,node_contracts,node_config toneAmber
class node_perception,node_safety,node_navigation,node_emergency,node_fall,node_sos,node_phrasing toneMint
class node_registry,node_providers,node_detector,node_router,node_asr toneRose
class node_net,node_main,node_ui,node_speech,node_navigation_ui,node_voice,node_map toneIndigo
```

## Components

### Frame capture

- [Frame sender](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/sender.js): `web/js/sender.js`

### Backend transport

- [Application setup](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/main.py): `app/main.py`
- [HTTP and WS routes](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/routes.py): `app/api/routes.py`
- [Frame upload](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/uploads.py): `app/api/uploads.py`
- [ASR endpoint](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/speech.py): `app/api/speech.py`
- [Announcement envelope](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/envelope.py): `app/api/envelope.py`
- [Broadcast hub](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/api/hub.py): `app/api/hub.py`
- [Announcement gate](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/arbiter.py): `app/core/arbiter.py`
- [Frame contracts](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/contracts.py): `app/contracts.py`
- [Runtime configuration](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/config.py): `app/config.py`

### Assistance logic

- [Scene perception](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/layers/perception.py): `app/core/layers/perception.py`
- [Safety analysis](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/layers/safety.py): `app/core/layers/safety.py`
- [Route planning](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/layers/navigation.py): `app/core/layers/navigation.py`
- [Emergency states](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/layers/emergency.py): `app/core/layers/emergency.py`
- [Fall detection](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/rules/fall.py): `app/core/rules/fall.py`
- [SOS escalation](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/rules/sos.py): `app/core/rules/sos.py`
- [Announcement phrasing](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/rules/phrasing.py): `app/core/rules/phrasing.py`

### Providers and routing

- [Implementation registry](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/registry.py): `app/core/registry.py`
- [Vision providers](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/providers/openai_compat.py): `app/core/providers/openai_compat.py`
- [Object detector](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/detectors/qwen_vl.py): `app/core/detectors/qwen_vl.py`
- [Map router](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/routers/baidu.py): `app/core/routers/baidu.py`
- [Speech recognizer](https://github.com/shuishen-cloud/blinderdetector/blob/main/app/core/asr/dashscope.py): `app/core/asr/dashscope.py`

### User experience

- [Client transport](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/net.js): `web/js/net.js`
- [App assembly](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/main.js): `web/js/main.js`
- [Announcement UI](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/ui.js): `web/js/ui.js`
- [Speech and haptics](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/speech.js): `web/js/speech.js`
- [Destination input](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/nav.js): `web/js/nav.js`
- [Voice input](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/js/voice.js): `web/js/voice.js`
- [Route display](https://github.com/shuishen-cloud/blinderdetector/blob/main/web/map.js): `web/map.js`

### Other components

- **User**
- **Browser client**

## Connections

- User → Frame sender: provides frames
- Frame sender → Client transport: uploads frames (evidence: `web/js/sender.js`)
- Client transport → Frame upload: POSTs frames
- Frame upload → Announcement envelope: builds response (evidence: `app/api/routes.py`)
- Application setup → HTTP and WS routes: builds routes (evidence: `app/main.py`)
- Application setup → Implementation registry: loads implementations (evidence: `app/main.py`)
- Application setup → Broadcast hub: creates hub (evidence: `app/main.py`)
- Application setup → Announcement envelope: creates envelope (evidence: `app/main.py`)
- Application setup → Announcement gate: creates gate (evidence: `app/main.py`)
- HTTP and WS routes → Frame upload: routes uploads (evidence: `app/api/routes.py`)
- HTTP and WS routes → ASR endpoint: routes audio (evidence: `app/api/routes.py`)
- HTTP and WS routes → Announcement envelope: routes requests (evidence: `app/api/routes.py`)
- HTTP and WS routes → Broadcast hub: serves stream (evidence: `app/api/routes.py`)
- Announcement envelope → Announcement gate: filters announcements (evidence: `app/api/envelope.py`)
- Announcement envelope → Broadcast hub: broadcasts announcements (evidence: `app/api/envelope.py`)
- Announcement envelope → Frame contracts: uses Frame (evidence: `app/api/envelope.py`)
- Announcement envelope → Scene perception: invokes layer
- Announcement envelope → Safety analysis: invokes layer
- Announcement envelope → Route planning: invokes layer
- Announcement envelope → Emergency states: invokes layer
- Scene perception → Vision providers: uses vision model
- Safety analysis → Object detector: detects obstacles
- Safety analysis → Announcement phrasing: phrases warnings (evidence: `app/core/layers/safety.py`)
- Route planning → Map router: requests routes
- Route planning → Announcement phrasing: phrases guidance
- Emergency states → Fall detection: tracks falls
- Emergency states → SOS escalation: advances SOS
- ASR endpoint → Implementation registry: selects recognizer (evidence: `app/api/speech.py`)
- ASR endpoint → Speech recognizer: transcribes audio
- Runtime configuration → Implementation registry: selects implementations (evidence: `app/config.py`)
- Browser client → Client transport: uses client transport
- Client transport → Broadcast hub: receives WS stream
- App assembly → Client transport: connects stream (evidence: `web/js/main.js`)
- App assembly → Announcement UI: initializes UI (evidence: `web/js/main.js`)
- App assembly → Destination input: initializes navigation (evidence: `web/js/main.js`)
- App assembly → Voice input: initializes voice (evidence: `web/js/main.js`)
- App assembly → Route display: initializes map (evidence: `web/js/main.js`)
- Client transport → Announcement UI: delivers announcements (evidence: `web/js/main.js`)
- Announcement UI → Speech and haptics: speaks announcements (evidence: `web/js/ui.js`)
- Voice input → Client transport: uploads audio (evidence: `web/js/voice.js`)
- Voice input → Destination input: submits destination (evidence: `web/js/voice.js`)
- Destination input → Client transport: requests route (evidence: `web/js/nav.js`)
- App assembly → Route display: forwards announcements (evidence: `web/js/main.js`)