# wiringai/mod_earshot architecture

> Architecture diagram of the GitHub repository wiringai/mod_earshot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wiringai/mod_earshot
- Repository: https://github.com/wiringai/mod_earshot

## Overview

Earshot is a FreeSWITCH module that bridges call media and control with AI agents over WebSockets. A dialplan application starts a stream; the module configures a protocol adapter and transport, sends caller audio, and plays decoded agent audio back into the call. The same session supports turn detection, barge-in, agent-issued call actions, masking, fan-out, and typed events and metrics. The graph reflects documented capabilities, but source excerpts are partial, so uncertain internal wiring—especially fan-out and masking—is not drawn.

Main entry points: [`src/mod_earshot.c`](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c) (FreeSWITCH Module), [`src/es_proto.c`](https://github.com/wiringai/mod_earshot/blob/main/src/es_proto.c) (Protocol Adapters), [`src/es_ws.c`](https://github.com/wiringai/mod_earshot/blob/main/src/es_ws.c) (WebSocket Client), [`src/es_codec.c`](https://github.com/wiringai/mod_earshot/blob/main/src/es_codec.c) (Audio Codecs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_call["Call Integration"]
  node_module["FreeSWITCH Module<br/>[mod_earshot.c]"]
end

subgraph group_media["Media Processing"]
  node_codec["Audio Codecs<br/>[es_codec.c]"]
  node_protobuf["Pipecat Protobuf<br/>[es_pb.c]"]
  node_turns["VAD and Turns<br/>[mod_earshot.c]"]
  node_playback["Playback Buffer<br/>[mod_earshot.c]"]
  node_fanout["Audio Fan-out<br/>[mod_earshot.c]"]
  node_masking["PCI / PII Masking<br/>[mod_earshot.c]"]
end

subgraph group_transport["Agent Transport"]
  node_proto["Protocol Adapters<br/>[es_proto.c]"]
  node_ws["WebSocket Client<br/>[es_ws.c]"]
end

subgraph group_ops["Control and Observability"]
  node_commands["Command Dispatch<br/>[mod_earshot.c]"]
  node_guard["Command Guards<br/>[es_cmdguard.c]"]
  node_events["Call Events<br/>[mod_earshot.c]"]
  node_metrics["Latency Metrics<br/>[mod_earshot.c]"]
end

node_dialplan(("Dialplan / Caller"))
node_agent(("AI Agent"))
node_esl(("ESL Subscriber"))

node_dialplan -->|"starts stream"| node_module
node_module -->|"starts connection"| node_ws
node_module -->|"creates adapter"| node_proto
node_module -->|"detects speech"| node_turns
node_module -->|"masks media"| node_masking
node_module -->|"fans out audio"| node_fanout
node_module -->|"converts media"| node_codec
node_proto -->|"sends frames"| node_ws
node_ws -->|"exchanges frames"| node_agent
node_agent -->|"returns audio / control"| node_ws
node_ws -->|"delivers callbacks"| node_module
node_module -->|"parses inbound data"| node_proto
node_proto -->|"encodes / decodes"| node_codec
node_proto -->|"handles Pipecat frames"| node_protobuf
node_module -->|"queues agent audio"| node_playback
node_playback -->|"supplies call audio"| node_module
node_module -->|"dispatches actions"| node_commands
node_commands -->|"validates arguments"| node_guard
node_commands -->|"executes call APIs"| node_module
node_module -->|"fires typed events"| node_events
node_turns -->|"reports turn events"| node_events
node_commands -->|"audits actions"| node_events
node_module -->|"collects counters"| node_metrics
node_metrics -->|"publishes metrics"| node_events
node_events -->|"delivers events"| node_esl

click node_module "https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c"
click node_proto "https://github.com/wiringai/mod_earshot/blob/main/src/es_proto.c"
click node_ws "https://github.com/wiringai/mod_earshot/blob/main/src/es_ws.c"
click node_codec "https://github.com/wiringai/mod_earshot/blob/main/src/es_codec.c"
click node_protobuf "https://github.com/wiringai/mod_earshot/blob/main/src/es_pb.c"
click node_turns "https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c"
click node_playback "https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c"
click node_fanout "https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c"
click node_masking "https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c"
click node_commands "https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c"
click node_guard "https://github.com/wiringai/mod_earshot/blob/main/src/es_cmdguard.c"
click node_events "https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c"
click node_metrics "https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_module toneBlue
class node_codec,node_protobuf,node_turns,node_playback,node_fanout,node_masking toneAmber
class node_proto,node_ws toneMint
class node_commands,node_guard,node_events,node_metrics toneRose
class node_dialplan,node_agent,node_esl toneIndigo
```

## Components

### Call Integration

- [FreeSWITCH Module](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c): `src/mod_earshot.c`

### Media Processing

- [Audio Codecs](https://github.com/wiringai/mod_earshot/blob/main/src/es_codec.c): `src/es_codec.c`
- [Pipecat Protobuf](https://github.com/wiringai/mod_earshot/blob/main/src/es_pb.c): `src/es_pb.c`
- [VAD and Turns](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c): `src/mod_earshot.c`
- [Playback Buffer](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c): `src/mod_earshot.c`
- [Audio Fan-out](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c): `src/mod_earshot.c`
- [PCI / PII Masking](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c): `src/mod_earshot.c`

### Agent Transport

- [Protocol Adapters](https://github.com/wiringai/mod_earshot/blob/main/src/es_proto.c): `src/es_proto.c`
- [WebSocket Client](https://github.com/wiringai/mod_earshot/blob/main/src/es_ws.c): `src/es_ws.c`

### Control and Observability

- [Command Dispatch](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c): `src/mod_earshot.c`
- [Command Guards](https://github.com/wiringai/mod_earshot/blob/main/src/es_cmdguard.c): `src/es_cmdguard.c`
- [Call Events](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c): `src/mod_earshot.c`
- [Latency Metrics](https://github.com/wiringai/mod_earshot/blob/main/src/mod_earshot.c): `src/mod_earshot.c`

### Other components

- **Dialplan / Caller**
- **AI Agent**
- **ESL Subscriber**

## Connections

- Dialplan / Caller → FreeSWITCH Module: starts stream
- FreeSWITCH Module → WebSocket Client: starts connection
- FreeSWITCH Module → Protocol Adapters: creates adapter
- FreeSWITCH Module → VAD and Turns: detects speech
- FreeSWITCH Module → PCI / PII Masking: masks media
- FreeSWITCH Module → Audio Fan-out: fans out audio
- FreeSWITCH Module → Audio Codecs: converts media
- Protocol Adapters → WebSocket Client: sends frames
- WebSocket Client → AI Agent: exchanges frames
- AI Agent → WebSocket Client: returns audio / control
- WebSocket Client → FreeSWITCH Module: delivers callbacks
- FreeSWITCH Module → Protocol Adapters: parses inbound data
- Protocol Adapters → Audio Codecs: encodes / decodes
- Protocol Adapters → Pipecat Protobuf: handles Pipecat frames
- FreeSWITCH Module → Playback Buffer: queues agent audio
- Playback Buffer → FreeSWITCH Module: supplies call audio
- FreeSWITCH Module → Command Dispatch: dispatches actions
- Command Dispatch → Command Guards: validates arguments
- Command Dispatch → FreeSWITCH Module: executes call APIs
- FreeSWITCH Module → Call Events: fires typed events
- VAD and Turns → Call Events: reports turn events
- Command Dispatch → Call Events: audits actions
- FreeSWITCH Module → Latency Metrics: collects counters
- Latency Metrics → Call Events: publishes metrics
- Call Events → ESL Subscriber: delivers events