How matiasmoram/prosperai-challenge works
1. Overall shape
- The repo is organized around three runtime processes: the voice bot (bot.py / src/prosper), the FastAPI EHR service (src/prosper/ehr), and the optional Operator Console (src/prosper/console).
- A fourth major boundary is the eval/test harness (evals/, tester/, tests/), which exercises the bot and EHR without being part of the live call path.
2. Live call path
- The bot process connects the browser client to a conversational pipeline built on Pipecat, ElevenLabs STT/TTS, and OpenAI chat models.
- User audio enters the voice pipeline, becomes text, is routed by the dispatcher, then the response is synthesized back to audio for the browser client.
3. Conversation control layer
- src/prosper/dispatcher.py, flows.py, prompts.py, tools.py, and speculation.py implement the core control logic.
- The design is a hybrid finite-state machine with per-state tool whitelists: the model can only call tools allowed by the current state.
- Tool execution returns typed Result[Ok, Err] values; error codes are part of the public behavior and are asserted by tests/evals.
4. Clinical workflow states
- The bot is not a generic chat assistant; it follows a scripted clinical flow for booking, canceling, rescheduling, triage, confirmation, and handoff.
- The state machine is the main boundary between “free-form model reasoning” and “safe, bounded business actions.”
5. EHR service boundary
- The EHR lives in a separate FastAPI process on port 8000 and is the authoritative system for patients, slots, and appointments.
- It exposes patient lookup, availability, create appointment, cancel appointment, and health endpoints.
- Persistence is via SQLite + SQLAlchemy; seeded slots and database constraints are used to enforce idempotency and prevent duplicate booking races.
6. Data access inside the EHR
- The EHR is split into models, schemas, repository, db, and api, which cleanly separates transport, validation, and persistence.
- This separation is important for the architecture diagram: the bot never talks to the database directly; it only goes through the EHR HTTP API via src/prosper/ehr_client.py.
7. External integrations
- The bot depends on two external AI services: ElevenLabs for speech and OpenAI for the dispatcher LLM and eval judge.
- src/prosper/integrations/ contains auxiliary integrations and a routing layer; these appear to support additional side systems beyond the core EHR workflow, but they are not the primary booking path.
8. Observability and audit
- src/prosper/observability/ handles timing and PII redaction.
- The console subsystem publishes an event stream and also writes an append-only JSONL audit log per session under data/audit/.
- The audit boundary is intentionally redacted at event emission, so logs are safe to replay/share without a second scrub pass.
9. Operator Console
- The console is a separate FastAPI/SSE surface on port 7861 that subscribes to live bot events.
- It shows FSM state, tool calls, latencies, transcript, patient identity with PII masked, offered slots, and final outcome.
- Architecturally, this is a read-only monitoring plane fed from the bot’s event bus, not a control plane.
10. Evaluation system
- evals/ is a first-class subsystem with scenario generation, scripted runs, a deterministic mock LLM, a judge, and trace replay.
- Every eval checks both state assertions and an LLM judge, so correctness is not tied to only one validation strategy.
- Live evals can mount the EHR in-process with httpx.ASGITransport, which keeps the harness hermetic while preserving the HTTP boundary semantics.
11. Human-simulation and adversarial testing
- tester/ provides persona models, noise injection, clarification behavior, and receipt-gate logic to simulate messy call behavior.
- tests/adversarial/ and the rest of tests/ probe routing, redaction, invariants, concurrency, transport errors, and edge-case conversation behavior.
- This repo is therefore test-driven around the architecture boundaries: dispatcher, EHR, observability, console, and eval harness.
12. Deployment and operations
- The live system is typically run as two local processes (make ehr and make bot), or together via Docker Compose.
- CI is built around linting, type checking, unit tests, and eval coverage, with a separate offline mock-eval path to avoid external API cost.
- The main architectural takeaway is that the project is a voice-agent orchestration system with a separate clinical backend, plus an observability/eval layer designed to validate behavior end-to-end.
Main entry points: dispatcher.py Dispatcher · bot.py Agent core · runner.py Eval runner · api.py EHR service
Components 23
Live call path
Runtime bot process that handles the browser voice call and conversation control.
- Bot runtime · Entry process for the live voice agent; wires the conversation pipeline together.
bot.py - Agent core · Main bot orchestration module used by the runtime.
src/prosper/bot.py - Dispatcher · Hybrid finite-state routing and tool permissioning layer.
src/prosper/dispatcher.py - Flows · Clinical workflow states and transitions for booking, triage, cancel, reschedule, and handoff.
src/prosper/flows.py - Prompts · System and state-specific instructions for the model.
src/prosper/prompts.py - Tools · Bounded business actions with Result-style success and error outcomes.
src/prosper/tools.py - Speculation · Support for speculative routing and early intent handling.
src/prosper/speculation.py - EHR client · Bot-side HTTP boundary to the clinical service; no direct database access.
src/prosper/ehr_client.py
Clinical backend
Separate FastAPI EHR service and its data access layers.
- EHR service · Separate clinical API process serving patients, slots, and appointments.
src/prosper/ehr/api.py - Repository · Data access layer that mediates all database operations.
src/prosper/ehr/repository.py - Models · Persistent entities and constraints for appointments and related records.
src/prosper/ehr/models.py - Schemas · Transport and validation models for the EHR API.
src/prosper/ehr/schemas.py - SQLite DB · Database setup and engine/session management for the EHR.
src/prosper/ehr/db.py
Operator console
Read-only monitoring plane fed by bot events and audit streams.
- Console server · FastAPI/SSE surface for live monitoring and session views.
src/prosper/console/server.py - Event bus · In-process event distribution from the bot to console subscribers.
src/prosper/console/bus.py - Audit log · Append-only, redacted per-session audit trail.
src/prosper/console/audit.py
Eval and test harness
Scenario, simulation, and adversarial validation around the live architecture.
- Eval runner · Scenario execution and scripted evaluation entrypoint.
evals/runner.py - Sim runner · Deterministic simulation and mocked live-call execution.
evals/sim.py - Judge · LLM-based scoring alongside state assertions.
evals/judge.py - Human sim · Adversarial call simulation with noise, clarification, and receipt-gate behavior.
tester
Support systems
Shared libraries for LLM access, observability, and extra integrations.
- LLM adapter · Shared access layer for chat and judge models.
src/prosper/llm.py - Observability · Timing and PII redaction utilities used across live and console paths.
src/prosper/observability - Integrations · Additional side integrations and routing beyond the core booking path.
src/prosper/integrations
Connections 25
- Bot runtime → Agent core: starts · Main process loads the bot orchestration layer.
- Agent core → Dispatcher: uses · Bot passes conversation control to the dispatcher.
- Dispatcher → Flows: advances · FSM states govern the clinical conversation flow.
- Dispatcher → Prompts: selects · State determines the active prompt frame.
- Dispatcher → Tools: permits · Only tools allowed by the current state are exposed.
- Dispatcher → Speculation: prefetches · Speculative logic supports early routing decisions.
- Dispatcher → LLM adapter: calls · Model reasoning is mediated by the shared LLM adapter.
- Tools → EHR client: invokes · Clinical actions are executed through the HTTP client.
- EHR client → EHR service: requests · Bot talks to the EHR only over HTTP.
- EHR service → Schemas: validates · API transport uses typed schemas.
- EHR service → Repository: delegates · Request handlers hand off to repository logic.
- Repository → Models: persists · Repository operates on SQLAlchemy models and constraints.
- Repository → SQLite DB: stores · Repository uses the SQLite session/engine layer.
- Agent core → Event bus: emits events · Live execution publishes redacted events for observers.
- Event bus → Console server: streams · Console subscribes to the event bus over SSE.
- Event bus → Audit log: records · The same events are appended to the audit log.
- Console server → Observability: redacts · Console presentation relies on shared redaction utilities.
- Agent core → Observability: measures · Timing and PII handling are shared across the live path.
- Eval runner → Sim runner: executes · Runner drives simulated and scripted scenarios.
- Eval runner → Judge: scores · Each eval combines state checks with an LLM judge.
- Eval runner → LLM adapter: uses · Evaluation depends on the shared model adapter.
- Human sim → Sim runner: feeds · Human-simulation behavior is plugged into simulated runs.
- Human sim → Dispatcher: stresses · Adversarial scenarios probe routing and safety boundaries.
- Integrations → Agent core: extends · Auxiliary integrations support side workflows around the core bot.
- Integrations → Observability: logs · Integration surfaces share redaction and timing conventions.