# ethanzhou22455663/her-project architecture

> Architecture diagram of the GitHub repository ethanzhou22455663/her-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ethanzhou22455663/her-project
- Repository: https://github.com/ethanzhou22455663/her-project

## Overview

Her is a React Native voice companion with push-to-talk and local wake-word listening. The main workflow is: the user starts listening, the wake lifecycle routes microphone frames to on-device keyword spotting, and a hit opens a realtime voice session; transcripts are archived and a rolling summary is refreshed for later sessions. Push mode starts the realtime session directly. Audio capture, playback, and keepalive support these flows. The README is empty; links reflect the shown source and SOURCE INDEX, while unsampled implementation details are omitted.

Main entry points: [`App.tsx`](https://github.com/ethanzhou22455663/her-project/blob/main/App.tsx) (Voice screen), [`src/realtime/client.ts`](https://github.com/ethanzhou22455663/her-project/blob/main/src/realtime/client.ts) (Realtime client), [`src/audio/capture.ts`](https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/capture.ts) (Audio capture), [`src/wake/lifecycle.ts`](https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/lifecycle.ts) (Wake lifecycle)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["App orchestration"]
  node_appscreen["Voice screen<br/>[App.tsx]"]
  node_lifecycle["Wake lifecycle<br/>[lifecycle.ts]"]
end

subgraph group_audio["Audio runtime"]
  node_capture["Audio capture<br/>[capture.ts]"]
  node_session["Audio session<br/>[session.ts]"]
  node_keepalive["Keepalive controls<br/>[keepalive.ts]"]
  node_playback["Audio playback<br/>[playback.ts]"]
end

subgraph group_wake["Wake detection"]
  node_engine["Wake engine<br/>[engine.ts]"]
  node_assets["Wake assets<br/>[assets.ts]"]
  node_keywords["Keyword builder<br/>[keywords.ts]"]
  node_models["ONNX model files"]
end

subgraph group_conversation["Conversation and memory"]
  node_client["Realtime client<br/>[client.ts]"]
  node_store[("Conversation memory<br/>[store.ts]")]
  node_summarizer["Memory summarizer<br/>[summarizer.ts]"]
end

subgraph group_configuration["Runtime configuration"]
  node_config["Runtime settings<br/>[index.ts]"]
  node_persona["Companion persona<br/>[persona.ts]"]
end

node_user(("User"))
node_realtime_service{{"DashScope realtime"}}
node_text_service{{"DashScope text API"}}
node_device[("Device audio and files")]
node_kws_native{{"Sherpa KWS runtime"}}

node_user -->|"starts or stops"| node_appscreen
node_appscreen -->|"starts listening"| node_lifecycle
node_lifecycle -->|"starts and routes"| node_capture
node_capture -->|"prepares audio"| node_session
node_appscreen -->|"sets audio policy"| node_session
node_lifecycle -->|"updates notifications"| node_keepalive
node_appscreen -->|"registers controls"| node_keepalive
node_capture -->|"sends wake frames"| node_engine
node_lifecycle -->|"initializes and resets"| node_engine
node_engine -->|"stages model"| node_assets
node_assets -->|"builds keywords"| node_keywords
node_assets -->|"copies model files"| node_models
node_engine -->|"initializes and feeds"| node_kws_native
node_appscreen -->|"opens realtime session"| node_client
node_capture -->|"streams session audio"| node_client
node_client -->|"sends and receives"| node_realtime_service
node_client -->|"plays audio response"| node_playback
node_appscreen -->|"stops playback"| node_playback
node_appscreen -->|"archives transcripts"| node_store
node_appscreen -->|"updates summary"| node_summarizer
node_client -->|"loads summary"| node_store
node_summarizer -->|"reads and writes memory"| node_store
node_summarizer -->|"requests summary"| node_text_service
node_store -->|"reads and writes files"| node_device
node_capture -->|"uses audio settings"| node_config
node_client -->|"uses session settings"| node_config
node_summarizer -->|"uses summary settings"| node_config
node_config -->|"imports persona"| node_persona

click node_appscreen "https://github.com/ethanzhou22455663/her-project/blob/main/App.tsx"
click node_lifecycle "https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/lifecycle.ts"
click node_capture "https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/capture.ts"
click node_session "https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/session.ts"
click node_keepalive "https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/keepalive.ts"
click node_playback "https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/playback.ts"
click node_engine "https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/engine.ts"
click node_assets "https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/assets.ts"
click node_keywords "https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/keywords.ts"
click node_models "https://github.com/ethanzhou22455663/her-project/tree/main/assets/wake/model"
click node_client "https://github.com/ethanzhou22455663/her-project/blob/main/src/realtime/client.ts"
click node_store "https://github.com/ethanzhou22455663/her-project/blob/main/src/memory/store.ts"
click node_summarizer "https://github.com/ethanzhou22455663/her-project/blob/main/src/memory/summarizer.ts"
click node_config "https://github.com/ethanzhou22455663/her-project/blob/main/src/config/index.ts"
click node_persona "https://github.com/ethanzhou22455663/her-project/blob/main/src/config/persona.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_appscreen,node_lifecycle,node_user toneBlue
class node_capture,node_session,node_keepalive,node_playback,node_device toneAmber
class node_engine,node_assets,node_keywords,node_models,node_text_service toneMint
class node_client,node_store,node_summarizer toneRose
class node_config,node_persona,node_realtime_service,node_kws_native toneIndigo
```

## Components

### App orchestration

- [Voice screen](https://github.com/ethanzhou22455663/her-project/blob/main/App.tsx): `App.tsx`
- [Wake lifecycle](https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/lifecycle.ts): `src/wake/lifecycle.ts`

### Audio runtime

- [Audio capture](https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/capture.ts): `src/audio/capture.ts`
- [Audio session](https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/session.ts): `src/audio/session.ts`
- [Keepalive controls](https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/keepalive.ts): `src/audio/keepalive.ts`
- [Audio playback](https://github.com/ethanzhou22455663/her-project/blob/main/src/audio/playback.ts): `src/audio/playback.ts`

### Wake detection

- [Wake engine](https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/engine.ts): `src/wake/engine.ts`
- [Wake assets](https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/assets.ts): `src/wake/assets.ts`
- [Keyword builder](https://github.com/ethanzhou22455663/her-project/blob/main/src/wake/keywords.ts): `src/wake/keywords.ts`
- [ONNX model files](https://github.com/ethanzhou22455663/her-project/tree/main/assets/wake/model): `assets/wake/model`

### Conversation and memory

- [Realtime client](https://github.com/ethanzhou22455663/her-project/blob/main/src/realtime/client.ts): `src/realtime/client.ts`
- [Conversation memory](https://github.com/ethanzhou22455663/her-project/blob/main/src/memory/store.ts): `src/memory/store.ts`
- [Memory summarizer](https://github.com/ethanzhou22455663/her-project/blob/main/src/memory/summarizer.ts): `src/memory/summarizer.ts`

### Runtime configuration

- [Runtime settings](https://github.com/ethanzhou22455663/her-project/blob/main/src/config/index.ts): `src/config/index.ts`
- [Companion persona](https://github.com/ethanzhou22455663/her-project/blob/main/src/config/persona.ts): `src/config/persona.ts`

### Other components

- **User**
- **DashScope realtime**
- **DashScope text API**
- **Device audio and files**
- **Sherpa KWS runtime**

## Connections

- User → Voice screen: starts or stops (evidence: `App.tsx`)
- Voice screen → Wake lifecycle: starts listening (evidence: `App.tsx`)
- Wake lifecycle → Audio capture: starts and routes (evidence: `src/wake/lifecycle.ts`)
- Audio capture → Audio session: prepares audio (evidence: `src/audio/capture.ts`)
- Voice screen → Audio session: sets audio policy (evidence: `App.tsx`)
- Wake lifecycle → Keepalive controls: updates notifications (evidence: `src/wake/lifecycle.ts`)
- Voice screen → Keepalive controls: registers controls (evidence: `App.tsx`)
- Audio capture → Wake engine: sends wake frames (evidence: `src/wake/lifecycle.ts`)
- Wake lifecycle → Wake engine: initializes and resets (evidence: `src/wake/lifecycle.ts`)
- Wake engine → Wake assets: stages model (evidence: `src/wake/engine.ts`)
- Wake assets → Keyword builder: builds keywords (evidence: `src/wake/assets.ts`)
- Wake assets → ONNX model files: copies model files (evidence: `src/wake/assets.ts`)
- Wake engine → Sherpa KWS runtime: initializes and feeds (evidence: `src/wake/engine.ts`)
- Voice screen → Realtime client: opens realtime session (evidence: `App.tsx`)
- Audio capture → Realtime client: streams session audio (evidence: `App.tsx`)
- Realtime client → DashScope realtime: sends and receives (evidence: `src/realtime/client.ts`)
- Realtime client → Audio playback: plays audio response (evidence: `src/realtime/client.ts`)
- Voice screen → Audio playback: stops playback (evidence: `App.tsx`)
- Voice screen → Conversation memory: archives transcripts (evidence: `App.tsx`)
- Voice screen → Memory summarizer: updates summary (evidence: `App.tsx`)
- Realtime client → Conversation memory: loads summary (evidence: `src/realtime/client.ts`)
- Memory summarizer → Conversation memory: reads and writes memory (evidence: `src/memory/summarizer.ts`)
- Memory summarizer → DashScope text API: requests summary (evidence: `src/memory/summarizer.ts`)
- Conversation memory → Device audio and files: reads and writes files (evidence: `src/memory/store.ts`)
- Audio capture → Runtime settings: uses audio settings (evidence: `src/audio/capture.ts`)
- Realtime client → Runtime settings: uses session settings (evidence: `src/realtime/client.ts`)
- Memory summarizer → Runtime settings: uses summary settings (evidence: `src/memory/summarizer.ts`)
- Runtime settings → Companion persona: imports persona (evidence: `src/config/index.ts`)
