How kushagragarg55/testaiagentpocwithllm works
This repository is a small agent-benchmarking pipeline: it runs three Groq-hosted customer-support models against shared scenarios and deterministic mock tools, records their traces, then scores those traces with DeepEval metrics and a separate judge. The default judge is local Ollama; a Groq-backed judge remains an optional alternative. The README documents the stage boundaries and outputs. The evaluation and judge implementations are excerpted; the pipeline links below are supported by those excerpts or explicit README statements.
Main entry points: generate_responses.py Agent runner · evaluate.py Trace evaluator · ollama_judge.py Ollama judge · groq_judge.py Groq judge
Components 13
Trace generation
- Support scenarios
data/scenarios.json - Agent runner
src/generate_responses.py - Mock support tools
src/generate_responses.py - Groq agent models
- Agent traces
Scoring
- Trace evaluator
src/evaluate.py - Evaluation metrics
src/evaluate.py - Evaluation results
results/eval_results.md
Judge backends
- Ollama judge
src/ollama_judge.py - Groq judge
src/groq_judge.py
Other components
- Operator
- Ollama service
- Groq service
Connections 12
- Operator → Agent runner: runs generation
README.md - Operator → Trace evaluator: runs evaluation
README.md - Agent runner → Support scenarios: reads scenarios
README.md - Agent runner → Groq agent models: queries models
src/generate_responses.py - Agent runner → Mock support tools: executes tool calls
src/generate_responses.py - Agent runner → Agent traces: writes traces
README.md - Trace evaluator → Agent traces: reads traces
src/evaluate.py - Trace evaluator → Evaluation metrics: measures cases
src/evaluate.py - Trace evaluator → Ollama judge: configures judge
src/evaluate.py - Ollama judge → Ollama service: sends prompts
src/ollama_judge.py - Trace evaluator → Evaluation results: persists scores
README.md - Groq judge → Groq service: sends prompts
src/groq_judge.py