# kushagragarg55/testaiagentpocwithllm architecture

> Architecture diagram of the GitHub repository kushagragarg55/testaiagentpocwithllm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kushagragarg55/testaiagentpocwithllm
- Repository: https://github.com/kushagragarg55/testaiagentpocwithllm

## Overview

This repository is a small agent-benchmarking pipeline: it runs three Groq-hosted customer-support models against shared scenarios and deterministic mock tools, records their traces, then scores those traces with DeepEval metrics and a separate judge. The default judge is local Ollama; a Groq-backed judge remains an optional alternative. The README documents the stage boundaries and outputs. The evaluation and judge implementations are excerpted; the pipeline links below are supported by those excerpts or explicit README statements.

Main entry points: [`src/generate_responses.py`](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/generate_responses.py) (Agent runner), [`src/evaluate.py`](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/evaluate.py) (Trace evaluator), [`src/ollama_judge.py`](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/ollama_judge.py) (Ollama judge), [`src/groq_judge.py`](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/groq_judge.py) (Groq judge)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_generation["Trace generation"]
  node_scenarios["Support scenarios<br/>[scenarios.json]"]
  node_agent_runner["Agent runner"]
  node_mock_tools["Mock support tools"]
  node_groq_models{{"Groq agent models"}}
  node_traces["Agent traces"]
end

subgraph group_evaluation["Scoring"]
  node_evaluator["Trace evaluator<br/>[evaluate.py]"]
  node_metrics["Evaluation metrics<br/>[evaluate.py]"]
  node_eval_results["Evaluation results<br/>[eval_results.md]"]
end

subgraph group_judges["Judge backends"]
  node_ollama_judge["Ollama judge<br/>[ollama_judge.py]"]
  node_groq_judge["Groq judge<br/>[groq_judge.py]"]
end

node_operator(("Operator"))
node_ollama_service{{"Ollama service"}}
node_groq_service{{"Groq service"}}

node_operator -->|"runs generation"| node_agent_runner
node_operator -->|"runs evaluation"| node_evaluator
node_agent_runner -->|"reads scenarios"| node_scenarios
node_agent_runner -->|"queries models"| node_groq_models
node_agent_runner -->|"executes tool calls"| node_mock_tools
node_agent_runner -->|"writes traces"| node_traces
node_evaluator -->|"reads traces"| node_traces
node_evaluator -->|"measures cases"| node_metrics
node_evaluator -->|"configures judge"| node_ollama_judge
node_ollama_judge -->|"sends prompts"| node_ollama_service
node_evaluator -->|"persists scores"| node_eval_results
node_groq_judge -->|"sends prompts"| node_groq_service

click node_scenarios "https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/data/scenarios.json"
click node_agent_runner "https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/generate_responses.py"
click node_mock_tools "https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/generate_responses.py"
click node_evaluator "https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/evaluate.py"
click node_metrics "https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/evaluate.py"
click node_eval_results "https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/results/eval_results.md"
click node_ollama_judge "https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/ollama_judge.py"
click node_groq_judge "https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/groq_judge.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scenarios,node_agent_runner,node_mock_tools,node_groq_models,node_traces toneBlue
class node_evaluator,node_metrics,node_eval_results toneAmber
class node_ollama_judge,node_groq_judge toneMint
class node_operator,node_ollama_service,node_groq_service toneIndigo
```

## Components

### Trace generation

- [Support scenarios](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/data/scenarios.json): `data/scenarios.json`
- [Agent runner](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/generate_responses.py): `src/generate_responses.py`
- [Mock support tools](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/generate_responses.py): `src/generate_responses.py`
- **Groq agent models**
- **Agent traces**

### Scoring

- [Trace evaluator](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/evaluate.py): `src/evaluate.py`
- [Evaluation metrics](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/evaluate.py): `src/evaluate.py`
- [Evaluation results](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/results/eval_results.md): `results/eval_results.md`

### Judge backends

- [Ollama judge](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/ollama_judge.py): `src/ollama_judge.py`
- [Groq judge](https://github.com/kushagragarg55/testaiagentpocwithllm/blob/main/src/groq_judge.py): `src/groq_judge.py`

### Other components

- **Operator**
- **Ollama service**
- **Groq service**

## Connections

- Operator → Agent runner: runs generation (evidence: `README.md`)
- Operator → Trace evaluator: runs evaluation (evidence: `README.md`)
- Agent runner → Support scenarios: reads scenarios (evidence: `README.md`)
- Agent runner → Groq agent models: queries models (evidence: `src/generate_responses.py`)
- Agent runner → Mock support tools: executes tool calls (evidence: `src/generate_responses.py`)
- Agent runner → Agent traces: writes traces (evidence: `README.md`)
- Trace evaluator → Agent traces: reads traces (evidence: `src/evaluate.py`)
- Trace evaluator → Evaluation metrics: measures cases (evidence: `src/evaluate.py`)
- Trace evaluator → Ollama judge: configures judge (evidence: `src/evaluate.py`)
- Ollama judge → Ollama service: sends prompts (evidence: `src/ollama_judge.py`)
- Trace evaluator → Evaluation results: persists scores (evidence: `README.md`)
- Groq judge → Groq service: sends prompts (evidence: `src/groq_judge.py`)
