# monke-sniper/second-ear architecture

> Architecture diagram of the GitHub repository monke-sniper/second-ear, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/monke-sniper/second-ear
- Repository: https://github.com/monke-sniper/second-ear

## Overview

Second Ear is a live phone-scam detection web app for an older caller and a family member. The main workflow streams microphone audio to the server, transcribes chunks, scores the running transcript, applies alert and code-warning rules, then presents events and sends a family push. A separate replay/evaluation branch supports recorded calls and held-out comparisons. The graph preserves the documented explanation and privacy capabilities; some internal wiring is inferred only from README descriptions, so those links use README evidence.

Main entry points: [`server/pipeline.py`](https://github.com/monke-sniper/second-ear/blob/main/server/pipeline.py) (Call pipeline), [`server/live.py`](https://github.com/monke-sniper/second-ear/blob/main/server/live.py) (Live socket), [`server/explainer.py`](https://github.com/monke-sniper/second-ear/blob/main/server/explainer.py) (Scam explanation), [`server/app.py`](https://github.com/monke-sniper/second-ear/blob/main/server/app.py) (FastAPI app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_browser["Browser experience"]
  node_web_call["Audio client<br/>[call.js]"]
  node_web_worklet["Audio capture<br/>[audio-worklet.js]"]
  node_web_view["Risk display<br/>[view.js]"]
  node_web_app["Web app<br/>[app.js]"]
end

subgraph group_runtime["Live call processing"]
  node_api["FastAPI app<br/>[app.py]"]
  node_live["Live socket<br/>[live.py]"]
  node_chunker["Audio chunker<br/>[chunker.py]"]
  node_stt["Speech recognition<br/>[stt.py]"]
  node_pipeline["Call pipeline<br/>[pipeline.py]"]
  node_scorer["Risk scorer<br/>[scorer.py]"]
  node_contracts["Wire contracts<br/>[contracts.py]"]
end

subgraph group_response["Alerts and response"]
  node_alert_rules["Alert rules<br/>[alerts.py]"]
  node_code_warning["Code warning<br/>[danger.py]"]
  node_number_redaction["Number redaction<br/>[numbers.py]"]
  node_explainer["Scam explanation<br/>[explainer.py]"]
  node_prompt["Safe prompt"]
end

subgraph group_integrations["External services"]
  node_push["Push notifier<br/>[notify.py]"]
  node_anthropic["Anthropic API"]
  node_gemini_client["Gemini client<br/>[gemini_client.py]"]
  node_gemini_api["Gemini API"]
  node_ntfy["ntfy service"]
end

subgraph group_evaluation["Evaluation"]
  node_replay["Call replay<br/>[replay.py]"]
  node_eval_runner["Evaluation runner<br/>[run_eval.py]"]
  node_heldout["Held-out scoring<br/>[score_heldout.py]"]
  node_metrics["Metrics<br/>[metrics.py]"]
  node_keyword["Keyword baseline"]
end

node_caller(("Caller"))
node_family(("Family member"))

node_caller -->|"speaks near"| node_web_worklet
node_web_worklet -->|"streams audio"| node_web_call
node_web_call -->|"sends audio"| node_api
node_api -->|"routes socket"| node_live
node_live -->|"chunks audio"| node_chunker
node_live -->|"transcribes chunks"| node_stt
node_live -->|"processes text"| node_pipeline
node_pipeline -->|"scores transcript"| node_scorer
node_pipeline -->|"decides alerts"| node_alert_rules
node_pipeline -->|"checks number risk"| node_code_warning
node_pipeline -->|"enriches alert"| node_explainer
node_pipeline -->|"redacts outbound text"| node_prompt
node_live -->|"sends notification"| node_push
node_live -->|"emits events"| node_contracts
node_web_app -->|"uses audio client"| node_web_call
node_web_app -->|"renders events"| node_web_view
node_web_view -->|"shows risk alert"| node_caller
node_push -->|"publishes push"| node_ntfy
node_ntfy -->|"delivers alert"| node_family
node_explainer -->|"requests explanation"| node_anthropic
node_explainer -.->|"selects provider"| node_gemini_client
node_gemini_client -->|"calls API"| node_gemini_api
node_explainer -->|"redacts numbers"| node_number_redaction
node_api -->|"serves simulation"| node_replay
node_eval_runner -->|"replays calls"| node_replay
node_replay -->|"uses live pipeline"| node_pipeline
node_eval_runner -->|"summarizes results"| node_metrics
node_heldout -->|"compares baseline"| node_keyword
node_heldout -->|"computes scores"| node_metrics

click node_web_call "https://github.com/monke-sniper/second-ear/blob/main/web/call.js"
click node_web_worklet "https://github.com/monke-sniper/second-ear/blob/main/web/audio-worklet.js"
click node_web_view "https://github.com/monke-sniper/second-ear/blob/main/web/view.js"
click node_web_app "https://github.com/monke-sniper/second-ear/blob/main/web/app.js"
click node_api "https://github.com/monke-sniper/second-ear/blob/main/server/app.py"
click node_live "https://github.com/monke-sniper/second-ear/blob/main/server/live.py"
click node_chunker "https://github.com/monke-sniper/second-ear/blob/main/server/chunker.py"
click node_stt "https://github.com/monke-sniper/second-ear/blob/main/server/stt.py"
click node_pipeline "https://github.com/monke-sniper/second-ear/blob/main/server/pipeline.py"
click node_scorer "https://github.com/monke-sniper/second-ear/blob/main/server/scorer.py"
click node_contracts "https://github.com/monke-sniper/second-ear/blob/main/server/contracts.py"
click node_alert_rules "https://github.com/monke-sniper/second-ear/blob/main/server/alerts.py"
click node_code_warning "https://github.com/monke-sniper/second-ear/blob/main/server/danger.py"
click node_number_redaction "https://github.com/monke-sniper/second-ear/blob/main/server/numbers.py"
click node_explainer "https://github.com/monke-sniper/second-ear/blob/main/server/explainer.py"
click node_prompt "https://github.com/monke-sniper/second-ear/blob/main/server/explainer_prompt.py"
click node_push "https://github.com/monke-sniper/second-ear/blob/main/server/notify.py"
click node_gemini_client "https://github.com/monke-sniper/second-ear/blob/main/server/gemini_client.py"
click node_replay "https://github.com/monke-sniper/second-ear/blob/main/eval/replay.py"
click node_eval_runner "https://github.com/monke-sniper/second-ear/blob/main/eval/run_eval.py"
click node_heldout "https://github.com/monke-sniper/second-ear/blob/main/eval/score_heldout.py"
click node_metrics "https://github.com/monke-sniper/second-ear/blob/main/eval/metrics.py"
click node_keyword "https://github.com/monke-sniper/second-ear/blob/main/eval/keyword_baseline.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web_call,node_web_worklet,node_web_view,node_web_app toneBlue
class node_api,node_live,node_chunker,node_stt,node_pipeline,node_scorer,node_contracts toneAmber
class node_alert_rules,node_code_warning,node_number_redaction,node_explainer,node_prompt toneMint
class node_push,node_anthropic,node_gemini_client,node_gemini_api,node_ntfy toneRose
class node_replay,node_eval_runner,node_heldout,node_metrics,node_keyword,node_caller,node_family toneIndigo
```

## Components

### Browser experience

- [Audio client](https://github.com/monke-sniper/second-ear/blob/main/web/call.js): `web/call.js`
- [Audio capture](https://github.com/monke-sniper/second-ear/blob/main/web/audio-worklet.js): `web/audio-worklet.js`
- [Risk display](https://github.com/monke-sniper/second-ear/blob/main/web/view.js): `web/view.js`
- [Web app](https://github.com/monke-sniper/second-ear/blob/main/web/app.js): `web/app.js`

### Live call processing

- [FastAPI app](https://github.com/monke-sniper/second-ear/blob/main/server/app.py): `server/app.py`
- [Live socket](https://github.com/monke-sniper/second-ear/blob/main/server/live.py): `server/live.py`
- [Audio chunker](https://github.com/monke-sniper/second-ear/blob/main/server/chunker.py): `server/chunker.py`
- [Speech recognition](https://github.com/monke-sniper/second-ear/blob/main/server/stt.py): `server/stt.py`
- [Call pipeline](https://github.com/monke-sniper/second-ear/blob/main/server/pipeline.py): `server/pipeline.py`
- [Risk scorer](https://github.com/monke-sniper/second-ear/blob/main/server/scorer.py): `server/scorer.py`
- [Wire contracts](https://github.com/monke-sniper/second-ear/blob/main/server/contracts.py): `server/contracts.py`

### Alerts and response

- [Alert rules](https://github.com/monke-sniper/second-ear/blob/main/server/alerts.py): `server/alerts.py`
- [Code warning](https://github.com/monke-sniper/second-ear/blob/main/server/danger.py): `server/danger.py`
- [Number redaction](https://github.com/monke-sniper/second-ear/blob/main/server/numbers.py): `server/numbers.py`
- [Scam explanation](https://github.com/monke-sniper/second-ear/blob/main/server/explainer.py): `server/explainer.py`
- [Safe prompt](https://github.com/monke-sniper/second-ear/blob/main/server/explainer_prompt.py): `server/explainer_prompt.py`

### External services

- [Push notifier](https://github.com/monke-sniper/second-ear/blob/main/server/notify.py): `server/notify.py`
- **Anthropic API**
- [Gemini client](https://github.com/monke-sniper/second-ear/blob/main/server/gemini_client.py): `server/gemini_client.py`
- **Gemini API**
- **ntfy service**

### Evaluation

- [Call replay](https://github.com/monke-sniper/second-ear/blob/main/eval/replay.py): `eval/replay.py`
- [Evaluation runner](https://github.com/monke-sniper/second-ear/blob/main/eval/run_eval.py): `eval/run_eval.py`
- [Held-out scoring](https://github.com/monke-sniper/second-ear/blob/main/eval/score_heldout.py): `eval/score_heldout.py`
- [Metrics](https://github.com/monke-sniper/second-ear/blob/main/eval/metrics.py): `eval/metrics.py`
- [Keyword baseline](https://github.com/monke-sniper/second-ear/blob/main/eval/keyword_baseline.py): `eval/keyword_baseline.py`

### Other components

- **Caller**
- **Family member**

## Connections

- Caller → Audio capture: speaks near
- Audio capture → Audio client: streams audio
- Audio client → FastAPI app: sends audio
- FastAPI app → Live socket: routes socket (evidence: `server/app.py`)
- Live socket → Audio chunker: chunks audio (evidence: `server/live.py`)
- Live socket → Speech recognition: transcribes chunks (evidence: `server/live.py`)
- Live socket → Call pipeline: processes text (evidence: `server/live.py`)
- Call pipeline → Risk scorer: scores transcript (evidence: `server/pipeline.py`)
- Call pipeline → Alert rules: decides alerts (evidence: `server/pipeline.py`)
- Call pipeline → Code warning: checks number risk (evidence: `server/pipeline.py`)
- Call pipeline → Scam explanation: enriches alert (evidence: `server/pipeline.py`)
- Call pipeline → Safe prompt: redacts outbound text (evidence: `server/pipeline.py`)
- Live socket → Push notifier: sends notification (evidence: `server/live.py`)
- Live socket → Wire contracts: emits events (evidence: `server/live.py`)
- Web app → Audio client: uses audio client (evidence: `web/app.js`)
- Web app → Risk display: renders events (evidence: `web/app.js`)
- Risk display → Caller: shows risk alert
- Push notifier → ntfy service: publishes push (evidence: `server/notify.py`)
- ntfy service → Family member: delivers alert
- Scam explanation → Anthropic API: requests explanation (evidence: `server/explainer.py`)
- Scam explanation → Gemini client: selects provider (evidence: `server/explainer.py`)
- Gemini client → Gemini API: calls API (evidence: `server/gemini_client.py`)
- Scam explanation → Number redaction: redacts numbers
- FastAPI app → Call replay: serves simulation (evidence: `server/app.py`)
- Evaluation runner → Call replay: replays calls (evidence: `eval/run_eval.py`)
- Call replay → Call pipeline: uses live pipeline (evidence: `eval/replay.py`)
- Evaluation runner → Metrics: summarizes results (evidence: `eval/run_eval.py`)
- Held-out scoring → Keyword baseline: compares baseline (evidence: `eval/score_heldout.py`)
- Held-out scoring → Metrics: computes scores (evidence: `eval/score_heldout.py`)
