How confident-ai/deepeval works
DeepEval is a Python LLM evaluation framework for testing model applications, agent trajectories, and individual agent steps. Users define test cases or datasets, select metrics, and run evaluations to judge outputs and report results; models and local NLP back those judgments. Separate capabilities cover trace capture and inspection, synthetic data generation, conversation simulation, prompt optimization, and integrations with agent frameworks. The graph preserves these README-documented branches. Relationships are documented at a high level; unsampled implementation details and call paths are not inferred.
Main entry points: evaluate.py Evaluation runner · llm_test_case.py Test cases · dataset.py Datasets · metrics Evaluation metrics
Components 16
Evaluation workflow
- Test cases
deepeval/test_case/llm_test_case.py - Datasets
deepeval/dataset/dataset.py - Evaluation runner
deepeval/evaluate/evaluate.py
Judging and models
- Evaluation metrics
deepeval/metrics - Judge models
deepeval/models - Safety classifiers
deepeval/classifiers
Tracing and inspection
- Tracing
deepeval/tracing/tracing.py - Trace inspector
deepeval/inspect/loader.py
Data and optimization
- Synthetic data
deepeval/synthesizer/synthesizer.py - Conversation simulation
deepeval/simulator/conversation_simulator.py - Prompt optimization
deepeval/optimizer/prompt_optimizer.py
Configuration and integrations
- Framework integrations
deepeval/integrations - Settings
deepeval/config/settings.py - CLI
deepeval/cli/main.py - Confident AI
deepeval/confident/api.py
Other components
- Developer
Connections 16
- Developer → Test cases: defines tests
README.md - Developer → Datasets: prepares data
README.md - Test cases → Evaluation runner: evaluates
README.md - Datasets → Evaluation runner: evaluates
README.md - Evaluation runner → Evaluation metrics: scores with
README.md - Evaluation metrics → Judge models: judges with
README.md - Developer → Tracing: instruments apps
README.md - Framework integrations → Tracing: captures traces
README.md - Developer → Trace inspector: inspects traces
README.md - Developer → Synthetic data: generates data
README.md - Developer → Conversation simulation: simulates chats
README.md - Developer → Prompt optimization: optimizes prompts
README.md - Developer → CLI: configures
README.md - CLI → Settings: sets options
README.md - Developer → Safety classifiers: evaluates safety
README.md - Evaluation runner → Confident AI: shares results