How fixadev/fixa works
Project type and purpose
• This is a Python library (“fixa-dev”) for automating the testing and evaluation of voice agents. It’s not a web UI or monolithic app, but a modular test-orchestration framework that integrates with telephony (Twilio/ngrok), voice agent engines (Pipecat + TTS), transcription services (Deepgram), LLM evaluators (OpenAI or cloud), and telemetry/observability layers (fixa observe).
File structure
• Top-level:
• src/fixa/agent.py, bot.py, scenario.py, evaluation.py, test.py, utils.py – core domain objects
• src/fixa/test_runner/{server.py, service.py, views.py} – HTTP/websocket server and orchestration logic
• src/fixa/evaluators/{local.py, cloud.py, evaluator.py} – pluggable evaluation strategies
• src/fixa/telemetry/{service.py, views.py} – metrics, logs, UI views for fixa observe
• examples/ – three usage patterns (single, multiple, cloud) demonstrating how TestRunner is configured
• tests/ – unit and integration tests for each evaluator and orchestration flow
• config files: pyproject.toml, .github/workflows/python-publish.yml, .env examples
README for additional insights
• The workflow:
a. User defines Agent + Scenario + Evaluations
b. Spins up ngrok → TestRunner (asyncio) listens for Twilio webhooks
c. TestRunner instructs Twilio to place calls
d. Voice agent (Pipecat + Cartesia TTS) drives the call, audio passes through Twilio
e. Twilio records conversation → audio file retrieved
f. Deepgram transcribes → transcript sent to Evaluator (Local/OpenAI or Cloud API)
g. Results and telemetry emitted to console or fixa observe UI
Essential details
• Internal TestRunner.http/ws endpoints (server.py/views.py) and how ngrok maps to them
• That Agent Engine uses Pipecat under the hood
• Evaluator’s strategy interface (src/fixa/evaluators/evaluator.py)
• How telemetry.service.py and telemetry.views.py power the cloud UI