himanshu-1703/llmops-agent-app

How himanshu-1703/llmops-agent-app works

Hiring Agent is a workspace-based hiring assistant: a user submits a task, the API runs a guarded plan-and-execute agent over resume files and a candidate tracker, then returns the response and workspace changes. A frontend and CLI provide ways to run the agent. Separate evaluation and experimentation subsystems assess agent quality and manage prompt/config challengers through MLflow. Edges reflect excerpted calls or SOURCE INDEX references; unsampled frontend/API wiring and some operational details are not asserted.

Main entry points: agent.py Plan-Execute Agent · eval_agent.py Evaluation Runner · execute_experiment_pipeline.py Experiment Pipeline · clients.py LLM Clients

Components 25

User Interfaces

Agent Frontend src/hiring_agent/frontend/agent_app.py
API Client src/hiring_agent/frontend/api_client.py
FastAPI App src/hiring_agent/api/main.py
Agent Route src/hiring_agent/api/routers/agent.py
Workspace Route src/hiring_agent/api/routers/workspace.py
Health Route src/hiring_agent/api/routers/health.py
API Schemas src/hiring_agent/api/schemas.py

Hiring Agent Runtime

Plan-Execute Agent src/hiring_agent/app/agent.py
Hiring Guardrails src/hiring_agent/app/guards.py
Agent Tools src/hiring_agent/app/tools.py
Hiring Workspace src/hiring_agent/app/workspace.py
Prompt Loading src/hiring_agent/app/prompts.py
LLM Clients src/hiring_agent/app/clients.py
Agent Parameters src/hiring_agent/config/parameter_config.py

Evaluation

Evaluation Runner src/hiring_agent/evals/eval_agent.py
Evaluation Metrics src/hiring_agent/evals/metrics.py
Plan Quality Judge src/hiring_agent/evals/plan_judge.py
Plan Adherence Judge src/hiring_agent/evals/adherence_judge.py
Slim Trace src/hiring_agent/evals/slim_trace.py
Evaluation Reports src/hiring_agent/evals/report.py

Experimentation

Experiment Pipeline src/hiring_agent/experimentation/execute_experiment_pipeline.py
MLflow Tracking src/hiring_agent/utils/mlflow_utils.py
Promote Challenger src/hiring_agent/experimentation/promote_challenger.py
Reject Challenger src/hiring_agent/experimentation/reject_challenger.py

Other components

Hiring User

Connections 38

Hiring User → Agent Frontend: submits task
Agent Frontend → API Client: uses src/hiring_agent/frontend/agent_app.py
API Client → FastAPI App: sends requests
FastAPI App → Agent Route: registers route src/hiring_agent/api/main.py
FastAPI App → Workspace Route: registers route src/hiring_agent/api/main.py
FastAPI App → Health Route: registers route src/hiring_agent/api/main.py
Agent Route → API Schemas: uses models src/hiring_agent/api/routers/agent.py
Agent Route → Plan-Execute Agent: runs agent src/hiring_agent/api/routers/agent.py
Agent Route → Hiring Workspace: snapshots workspace src/hiring_agent/api/routers/agent.py
Workspace Route → Hiring Workspace: uses workspace src/hiring_agent/api/routers/workspace.py
Health Route → LLM Clients: checks clients src/hiring_agent/api/routers/health.py
Health Route → Hiring Guardrails: checks guards src/hiring_agent/api/routers/health.py
Plan-Execute Agent → Hiring Guardrails: validates stages src/hiring_agent/app/agent.py
Plan-Execute Agent → Agent Tools: invokes tools src/hiring_agent/app/agent.py
Plan-Execute Agent → Hiring Workspace: snapshots state src/hiring_agent/app/agent.py
Plan-Execute Agent → Prompt Loading: loads prompts src/hiring_agent/app/agent.py
Plan-Execute Agent → LLM Clients: uses models src/hiring_agent/app/agent.py
Agent Tools → Hiring Workspace: reads and writes src/hiring_agent/app/tools.py
Agent Tools → Hiring Guardrails: applies checks src/hiring_agent/app/tools.py
Agent Tools → LLM Clients: uses clients src/hiring_agent/app/tools.py
Prompt Loading → LLM Clients: uses logger src/hiring_agent/app/prompts.py
Prompt Loading → Agent Parameters: reads settings src/hiring_agent/app/prompts.py
LLM Clients → Agent Parameters: loads settings src/hiring_agent/app/clients.py
Evaluation Runner → Plan-Execute Agent: runs agent src/hiring_agent/evals/eval_agent.py
Evaluation Runner → Evaluation Metrics: builds metrics src/hiring_agent/evals/eval_agent.py
Evaluation Runner → Plan Quality Judge: judges plans src/hiring_agent/evals/eval_agent.py
Evaluation Runner → Plan Adherence Judge: judges adherence src/hiring_agent/evals/eval_agent.py
Evaluation Runner → Evaluation Reports: writes reports src/hiring_agent/evals/eval_agent.py
Evaluation Metrics → Slim Trace: wraps trace metrics src/hiring_agent/evals/metrics.py
Evaluation Metrics → Plan Quality Judge: creates judge src/hiring_agent/evals/metrics.py
Evaluation Metrics → Plan Adherence Judge: creates judge src/hiring_agent/evals/metrics.py
Experiment Pipeline → Evaluation Runner: runs evaluations src/hiring_agent/experimentation/execute_experiment_pipeline.py
Experiment Pipeline → Evaluation Reports: reads averages src/hiring_agent/experimentation/execute_experiment_pipeline.py
Experiment Pipeline → Prompt Loading: loads prompts src/hiring_agent/experimentation/execute_experiment_pipeline.py
Experiment Pipeline → Agent Parameters: uses parameters src/hiring_agent/experimentation/execute_experiment_pipeline.py
Experiment Pipeline → MLflow Tracking: logs runs src/hiring_agent/experimentation/execute_experiment_pipeline.py
Promote Challenger → MLflow Tracking: promotes run src/hiring_agent/experimentation/promote_challenger.py
Reject Challenger → MLflow Tracking: rejects run src/hiring_agent/experimentation/reject_challenger.py

Drawn by GitDiagram on November 10, 2026 from the repository's file tree, README and source files. Read as Markdown ·