How aymenfurter/gpt-4o-latency-comparison works
Project type and purpose
• This is a Python-based benchmarking toolkit with a simple Gradio “frontend” (app.py) and a modular “backend” consisting of an orchestrator (benchmark_runner.py), model-wrapper adapters (in models/), and utility modules (in utils/).
• It measures and compares latencies of various GPT-4o audio implementations by driving Azure OpenAI and Azure Speech APIs, analyzing returned audio via FFmpeg/pydub, and presenting metrics and audio samples in a web UI.
Main components
a. Gradio Interface (app.py)
b. Benchmark Orchestrator (benchmark_runner.py)
c. Model Adapter Layer (models/*.py)
d. Utility Layer (utils/audio_utils.py, utils/metrics.py, utils/client.py, etc.)
e. Configuration Loader (config.py + .env)
f. External Services:
• Azure OpenAI endpoints (GPT-4o realtime, audio preview, transcribe, mini-models)
• Azure Speech Services (speech-to-text, text-to-speech)
g. Local Audio Analysis Tools (FFmpeg/pydub binary invoked by audio_utils)
Relationships & data flow
1. User interacts with the Gradio Interface → selects prompt, models, iterations.
2. Gradio calls the Benchmark Orchestrator → loads config, instantiates each requested Model Adapter.
3. For each model and iteration:
• Adapter invokes Utils.client to call the appropriate Azure API.
• Azure returns text/audio streams.
• Adapter writes interim audio to disk or buffer.
• Utils.audio_utils calls FFmpeg/pydub to measure “time to playback” and “audio duration.”
• Utils.metrics aggregates timing data.
4. Orchestrator collects metrics & audio snippets → sends structured results back to Gradio.
5. Gradio renders charts, tables, and embeds audio players.
Architectural patterns & design principles
• Adapter Pattern for each GPT-4o variant (models/*)
• Single Orchestrator coordinating isolated benchmarks (benchmark_runner.py)
• Separation of concerns: UI (app.py) vs. Orchestration vs. Model logic vs. Utility functions
• Config-based deployment and environment credentials via config.py + .env
Detailed breakdown
– config.py reads .env
– benchmark_runner.py loop
5. Utils layer (audio_utils, metrics, client, exceptions) sits between adapters and local/external services.
7. Local audio analysis: FFmpeg/pydub invoked by audio_utils → returns durations.