How ethanzhou22455663/her-project works
Her is a React Native voice companion with push-to-talk and local wake-word listening. The main workflow is: the user starts listening, the wake lifecycle routes microphone frames to on-device keyword spotting, and a hit opens a realtime voice session; transcripts are archived and a rolling summary is refreshed for later sessions. Push mode starts the realtime session directly. Audio capture, playback, and keepalive support these flows. The README is empty; links reflect the shown source and SOURCE INDEX, while unsampled implementation details are omitted.
Main entry points: App.tsx Voice screen · client.ts Realtime client · capture.ts Audio capture · lifecycle.ts Wake lifecycle
Components 20
App orchestration
- Voice screen
App.tsx - Wake lifecycle
src/wake/lifecycle.ts
Audio runtime
- Audio capture
src/audio/capture.ts - Audio session
src/audio/session.ts - Keepalive controls
src/audio/keepalive.ts - Audio playback
src/audio/playback.ts
Wake detection
- Wake engine
src/wake/engine.ts - Wake assets
src/wake/assets.ts - Keyword builder
src/wake/keywords.ts - ONNX model files
assets/wake/model
Conversation and memory
- Realtime client
src/realtime/client.ts - Conversation memory
src/memory/store.ts - Memory summarizer
src/memory/summarizer.ts
Runtime configuration
- Runtime settings
src/config/index.ts - Companion persona
src/config/persona.ts
Other components
- User
- DashScope realtime
- DashScope text API
- Device audio and files
- Sherpa KWS runtime
Connections 28
- User → Voice screen: starts or stops
App.tsx - Voice screen → Wake lifecycle: starts listening
App.tsx - Wake lifecycle → Audio capture: starts and routes
src/wake/lifecycle.ts - Audio capture → Audio session: prepares audio
src/audio/capture.ts - Voice screen → Audio session: sets audio policy
App.tsx - Wake lifecycle → Keepalive controls: updates notifications
src/wake/lifecycle.ts - Voice screen → Keepalive controls: registers controls
App.tsx - Audio capture → Wake engine: sends wake frames
src/wake/lifecycle.ts - Wake lifecycle → Wake engine: initializes and resets
src/wake/lifecycle.ts - Wake engine → Wake assets: stages model
src/wake/engine.ts - Wake assets → Keyword builder: builds keywords
src/wake/assets.ts - Wake assets → ONNX model files: copies model files
src/wake/assets.ts - Wake engine → Sherpa KWS runtime: initializes and feeds
src/wake/engine.ts - Voice screen → Realtime client: opens realtime session
App.tsx - Audio capture → Realtime client: streams session audio
App.tsx - Realtime client → DashScope realtime: sends and receives
src/realtime/client.ts - Realtime client → Audio playback: plays audio response
src/realtime/client.ts - Voice screen → Audio playback: stops playback
App.tsx - Voice screen → Conversation memory: archives transcripts
App.tsx - Voice screen → Memory summarizer: updates summary
App.tsx - Realtime client → Conversation memory: loads summary
src/realtime/client.ts - Memory summarizer → Conversation memory: reads and writes memory
src/memory/summarizer.ts - Memory summarizer → DashScope text API: requests summary
src/memory/summarizer.ts - Conversation memory → Device audio and files: reads and writes files
src/memory/store.ts - Audio capture → Runtime settings: uses audio settings
src/audio/capture.ts - Realtime client → Runtime settings: uses session settings
src/realtime/client.ts - Memory summarizer → Runtime settings: uses summary settings
src/memory/summarizer.ts - Runtime settings → Companion persona: imports persona
src/config/index.ts