# hewei-24/srtp architecture

> Architecture diagram of the GitHub repository hewei-24/srtp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hewei-24/srtp
- Repository: https://github.com/hewei-24/srtp

## Overview

心晴 is a local-first student mental-health support platform. Its primary workflow is browser chat or speech input, optional camera-based emotion analysis, a psychology response, then synthesized audio and avatar video playback. A separate 心理润园 branch provides educational and self-help resources. The README documents capabilities beyond the sampled source; links for those and optional SadTalker integration are therefore limited to documented relationships.

Main entry points: [`app.py`](https://github.com/hewei-24/srtp/blob/main/app.py) (REST service), [`static/js/main.js`](https://github.com/hewei-24/srtp/blob/main/static/js/main.js) (Browser UI), [`static/js/api.js`](https://github.com/hewei-24/srtp/blob/main/static/js/api.js) (API client), [`static/js/modules/speech.js`](https://github.com/hewei-24/srtp/blob/main/static/js/modules/speech.js) (Speech input)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experience["User experience"]
  node_browser["Browser UI<br/>[main.js]"]
  node_api["API client<br/>[api.js]"]
  node_speech_input["Speech input<br/>[speech.js]"]
  node_camera["Camera emotion<br/>[camera.js]"]
  node_avatar_ui["Avatar playback<br/>[avatar.js]"]
end

subgraph group_service["Flask services"]
  node_flask["REST service<br/>[app.py]"]
  node_emotion["Face emotion<br/>[app.py]"]
  node_speech_recognition["Speech recognition<br/>[app.py]"]
  node_conversation["Conversation state<br/>[app.py]"]
end

subgraph group_generation["Response generation"]
  node_psych_agent["Psychology agent<br/>[app.py]"]
  node_local_model["Local Qwen model<br/>[app.py]"]
  node_tts["Edge TTS<br/>[app.py]"]
  node_audio_output["Generated audio<br/>[app.py]"]
end

subgraph group_wellbeing["Wellbeing resources"]
  node_garden["Psychology garden<br/>[app.py]"]
end

subgraph group_avatar["Avatar media"]
  node_avatar_service["Avatar management<br/>[app.py]"]
  node_uploads["Custom uploads<br/>[app.py]"]
  node_speaking_video["Speaking videos<br/>[gradio_demo.py]"]
  node_sadtalker["SadTalker synthesis<br/>[inference.py]"]
end

node_student(("Student"))
node_deepface["DeepFace runtime"]
node_edge_tts_service["Edge TTS engine"]

node_student -->|"uses"| node_browser
node_browser -->|"captures speech"| node_speech_input
node_speech_input -->|"provides text"| node_browser
node_browser -->|"starts analysis"| node_camera
node_camera -->|"requests emotion"| node_api
node_api -->|"calls API"| node_flask
node_flask -->|"dispatches"| node_emotion
node_emotion -->|"analyzes face"| node_deepface
node_flask -->|"dispatches"| node_speech_recognition
node_flask -->|"dispatches"| node_psych_agent
node_psych_agent -->|"generates reply"| node_local_model
node_psych_agent -->|"manages history"| node_conversation
node_flask -->|"synthesizes reply"| node_tts
node_tts -->|"generates audio"| node_edge_tts_service
node_tts -->|"writes audio"| node_audio_output
node_flask -->|"selects avatar"| node_avatar_service
node_avatar_service -->|"stores upload"| node_uploads
node_flask -->|"selects video"| node_speaking_video
node_browser -->|"submits message"| node_api
node_browser -->|"updates playback"| node_avatar_ui
node_api -->|"returns response"| node_browser
node_browser -->|"plays audio"| node_audio_output
node_student -->|"uses resources"| node_garden
node_flask -.->|"optional integration"| node_sadtalker

click node_browser "https://github.com/hewei-24/srtp/blob/main/static/js/main.js"
click node_api "https://github.com/hewei-24/srtp/blob/main/static/js/api.js"
click node_speech_input "https://github.com/hewei-24/srtp/blob/main/static/js/modules/speech.js"
click node_camera "https://github.com/hewei-24/srtp/blob/main/static/js/modules/camera.js"
click node_avatar_ui "https://github.com/hewei-24/srtp/blob/main/static/js/modules/avatar.js"
click node_flask "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_emotion "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_speech_recognition "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_conversation "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_psych_agent "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_local_model "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_tts "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_audio_output "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_garden "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_avatar_service "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_uploads "https://github.com/hewei-24/srtp/blob/main/app.py"
click node_speaking_video "https://github.com/hewei-24/srtp/blob/main/SadTalker/src/gradio_demo.py"
click node_sadtalker "https://github.com/hewei-24/srtp/blob/main/SadTalker/inference.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_api,node_speech_input,node_camera,node_avatar_ui toneBlue
class node_flask,node_emotion,node_speech_recognition,node_conversation toneAmber
class node_psych_agent,node_local_model,node_tts,node_audio_output toneMint
class node_garden toneRose
class node_avatar_service,node_uploads,node_speaking_video,node_sadtalker,node_student,node_deepface,node_edge_tts_service toneIndigo
```

## Components

### User experience

- [Browser UI](https://github.com/hewei-24/srtp/blob/main/static/js/main.js): `static/js/main.js`
- [API client](https://github.com/hewei-24/srtp/blob/main/static/js/api.js): `static/js/api.js`
- [Speech input](https://github.com/hewei-24/srtp/blob/main/static/js/modules/speech.js): `static/js/modules/speech.js`
- [Camera emotion](https://github.com/hewei-24/srtp/blob/main/static/js/modules/camera.js): `static/js/modules/camera.js`
- [Avatar playback](https://github.com/hewei-24/srtp/blob/main/static/js/modules/avatar.js): `static/js/modules/avatar.js`

### Flask services

- [REST service](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`
- [Face emotion](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`
- [Speech recognition](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`
- [Conversation state](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`

### Response generation

- [Psychology agent](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`
- [Local Qwen model](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`
- [Edge TTS](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`
- [Generated audio](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`

### Wellbeing resources

- [Psychology garden](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`

### Avatar media

- [Avatar management](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`
- [Custom uploads](https://github.com/hewei-24/srtp/blob/main/app.py): `app.py`
- [Speaking videos](https://github.com/hewei-24/srtp/blob/main/SadTalker/src/gradio_demo.py): `SadTalker/src/gradio_demo.py`
- [SadTalker synthesis](https://github.com/hewei-24/srtp/blob/main/SadTalker/inference.py): `SadTalker/inference.py`

### Other components

- **Student**
- **DeepFace runtime**
- **Edge TTS engine**

## Connections

- Student → Browser UI: uses
- Browser UI → Speech input: captures speech
- Speech input → Browser UI: provides text
- Browser UI → Camera emotion: starts analysis
- Camera emotion → API client: requests emotion
- API client → REST service: calls API
- REST service → Face emotion: dispatches
- Face emotion → DeepFace runtime: analyzes face
- REST service → Speech recognition: dispatches
- REST service → Psychology agent: dispatches
- Psychology agent → Local Qwen model: generates reply
- Psychology agent → Conversation state: manages history
- REST service → Edge TTS: synthesizes reply
- Edge TTS → Edge TTS engine: generates audio
- Edge TTS → Generated audio: writes audio
- REST service → Avatar management: selects avatar
- Avatar management → Custom uploads: stores upload
- REST service → Speaking videos: selects video
- Browser UI → API client: submits message
- Browser UI → Avatar playback: updates playback
- API client → Browser UI: returns response
- Browser UI → Generated audio: plays audio
- Student → Psychology garden: uses resources
- REST service → SadTalker synthesis: optional integration