# fixadev/fixa architecture

> Architecture diagram of the GitHub repository fixadev/fixa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fixadev/fixa
- Repository: https://github.com/fixadev/fixa

## Overview

##### Project type and purpose
- This is a Python library (“fixa-dev”) for automating the testing and evaluation of voice agents. It’s not a web UI or monolithic app, but a modular test-orchestration framework that integrates with telephony (Twilio/ngrok), voice agent engines (Pipecat + TTS), transcription services (Deepgram), LLM evaluators (OpenAI or cloud), and telemetry/observability layers (fixa observe).

##### File structure
- Top-level:
- src/fixa/agent.py, bot.py, scenario.py, evaluation.py, test.py, utils.py – core domain objects
- src/fixa/test_runner/{server.py, service.py, views.py} – HTTP/websocket server and orchestration logic
- src/fixa/evaluators/{local.py, cloud.py, evaluator.py} – pluggable evaluation strategies
- src/fixa/telemetry/{service.py, views.py} – metrics, logs, UI views for fixa observe
- examples/ – three usage patterns (single, multiple, cloud) demonstrating how TestRunner is configured
- tests/ – unit and integration tests for each evaluator and orchestration flow
- config files: pyproject.toml, .github/workflows/python-publish.yml, .env examples

##### README for additional insights
- The workflow:
a. User defines Agent + Scenario + Evaluations
b. Spins up ngrok → TestRunner (asyncio) listens for Twilio webhooks
c. TestRunner instructs Twilio to place calls
d. Voice agent (Pipecat + Cartesia TTS) drives the call, audio passes through Twilio
e. Twilio records conversation → audio file retrieved
f. Deepgram transcribes → transcript sent to Evaluator (Local/OpenAI or Cloud API)
g. Results and telemetry emitted to console or fixa observe UI

##### Essential details
- Internal TestRunner.http/ws endpoints (server.py/views.py) and how ngrok maps to them
- That Agent Engine uses Pipecat under the hood
- Evaluator’s strategy interface (src/fixa/evaluators/evaluator.py)
- How telemetry.service.py and telemetry.views.py power the cloud UI

## Diagram (Mermaid)

```mermaid
graph TB
    %% Domain Objects
    subgraph "Domain Objects"
        Agent["agent.py"]:::domain
        Scenario["scenario.py"]:::domain
        EvaluationDom["evaluation.py"]:::domain
        TestDom["test.py"]:::domain
        Bot["bot.py"]:::domain
        Utils["utils.py"]:::domain
    end

    %% Example Tests
    subgraph "Example Tests"
        Single["single_test\nmain.py"]:::example
        SingleEnv[".env.example"]:::example
        Multi["multiple_tests\nmain.py"]:::example
        MultiEnv[".env.example"]:::example
        CloudEx["cloud_evaluator\nmain.py"]:::example
        CloudEnv[".env.example"]:::example
    end

    Single --> SingleEnv
    Multi --> MultiEnv
    CloudEx --> CloudEnv

    %% TestRunner Service
    subgraph "TestRunner Service"
        TRServer["server.py\n(HTTP/WebSocket)"]:::core
        TRService["service.py\n(Orchestration)"]:::core
        TRViews["views.py\n(API Handlers)"]:::core
    end

    TRServer --> TRService
    TRViews --> TRService

    %% Agent Engine
    subgraph "Agent Engine"
        AE["Pipecat + Cartesia TTS"]:::core
    end

    %% Telephony Interface
    subgraph "Telephony Interface"
        NGrok["ngrok Tunnel"]:::ext
        Twilio["Twilio\nProgrammable Voice"]:::ext
    end

    TRService -->|"initiate call"| Twilio
    NGrok --> TRServer
    AE <--> Twilio

    %% Recording Storage
    subgraph "Recording Storage"
        Rec["Twilio Recordings"]:::ext
    end

    Twilio --> Rec
    TRService -->|"fetch recording URL"| Rec

    %% Transcription Service
    subgraph "Transcription Service"
        STT["Deepgram STT API"]:::ext
    end

    Rec -->|"audio file"| STT
    STT -->|"transcript"| TRService

    %% Evaluator Plugins
    subgraph "Evaluator Plugins"
        EvalIF["evaluator.py\n(Interface)"]:::core
        LocalEval["local.py\n(LocalEvaluator)"]:::core
        CloudEval["cloud.py\n(CloudEvaluator)"]:::core
    end

    TRService -->|"invoke evaluation"| EvalIF
    EvalIF --> LocalEval
    EvalIF --> CloudEval
    LocalEval --> TRService

    %% Telemetry / Observe
    subgraph "Telemetry / Observe"
        TelService["service.py\n(Telemetry Service)"]:::tel
        TelViews["views.py\n(UI Views)"]:::tel
        ObserveUI["fixa observe UI"]:::tel
    end

    CloudEval -->|"push metrics"| TelService
    TelService --> TelViews
    TelViews --> ObserveUI

    %% Console Output
    TRService -->|"console summary"| CLI["Console (CLI)"]:::ext

    %% Tests
    subgraph "Unit & Integration Tests"
        Test1["test_simple.py"]:::test
        Test2["test_complex.py"]:::test
        TestLocal["test_local_evaluator.py"]:::test
        TestCloud["test_cloud_evaluator.py"]:::test
        TestCancel["cancel_call.py"]:::test
    end

    %% CI/CD & Metadata
    CI["python-publish.yml"]:::infra
    Metadata["pyproject.toml"]:::infra

    %% Click Events
    click Agent "https://github.com/fixadev/fixa/blob/main/src/fixa/agent.py"
    click Scenario "https://github.com/fixadev/fixa/blob/main/src/fixa/scenario.py"
    click EvaluationDom "https://github.com/fixadev/fixa/blob/main/src/fixa/evaluation.py"
    click TestDom "https://github.com/fixadev/fixa/blob/main/src/fixa/test.py"
    click Bot "https://github.com/fixadev/fixa/blob/main/src/fixa/bot.py"
    click Utils "https://github.com/fixadev/fixa/blob/main/src/fixa/utils.py"
    click TRServer "https://github.com/fixadev/fixa/blob/main/src/fixa/test_runner/server.py"
    click TRService "https://github.com/fixadev/fixa/blob/main/src/fixa/test_runner/service.py"
    click TRViews "https://github.com/fixadev/fixa/blob/main/src/fixa/test_runner/views.py"
    click EvalIF "https://github.com/fixadev/fixa/blob/main/src/fixa/evaluators/evaluator.py"
    click LocalEval "https://github.com/fixadev/fixa/blob/main/src/fixa/evaluators/local.py"
    click CloudEval "https://github.com/fixadev/fixa/blob/main/src/fixa/evaluators/cloud.py"
    click TelService "https://github.com/fixadev/fixa/blob/main/src/fixa/telemetry/service.py"
    click TelViews "https://github.com/fixadev/fixa/blob/main/src/fixa/telemetry/views.py"
    click Single "https://github.com/fixadev/fixa/blob/main/examples/single_test/main.py"
    click SingleEnv "https://github.com/fixadev/fixa/blob/main/examples/single_test/.env.example"
    click Multi "https://github.com/fixadev/fixa/blob/main/examples/multiple_tests/main.py"
    click MultiEnv "https://github.com/fixadev/fixa/blob/main/examples/multiple_tests/.env.example"
    click CloudEx "https://github.com/fixadev/fixa/blob/main/examples/cloud_evaluator/main.py"
    click CloudEnv "https://github.com/fixadev/fixa/blob/main/examples/cloud_evaluator/.env.example"
    click CI "https://github.com/fixadev/fixa/blob/main/.github/workflows/python-publish.yml"
    click Metadata "https://github.com/fixadev/fixa/blob/main/pyproject.toml"
    click Test1 "https://github.com/fixadev/fixa/blob/main/tests/test_simple.py"
    click Test2 "https://github.com/fixadev/fixa/blob/main/tests/test_complex.py"
    click TestLocal "https://github.com/fixadev/fixa/blob/main/tests/test_local_evaluator.py"
    click TestCloud "https://github.com/fixadev/fixa/blob/main/tests/test_cloud_evaluator.py"
    click TestCancel "https://github.com/fixadev/fixa/blob/main/tests/cancel_call.py"

    %% Styles
    classDef domain fill:#DDEBF7,stroke:#4F81BD,color:#000
    classDef core fill:#E2EFDA,stroke:#548235,color:#000
    classDef ext fill:#FCE4D6,stroke:#C0504D,color:#000
    classDef tel fill:#E7E6E6,stroke:#7F7F7F,color:#000
    classDef example fill:#FFF2CC,stroke:#BF8F00,color:#000
    classDef test fill:#F3E2A9,stroke:#B45F06,color:#000
    classDef infra fill:#D9E1F2,stroke:#2F5496,color:#000
```
