How saranyadharani/uber-support-agent works
This repository is an Uber customer-support agent: a customer message enters through the chat UI or pipeline, is classified with intent and risk signals, passes through a deterministic escalation policy, and receives a reply grounded in similar resolved cases. The reply is drafted even when the policy recommends human review. Other substantial capabilities include thread reconstruction and retrieval-index preparation, three comparable intent classifiers, and a golden-set evaluation harness with reply judging and metrics. Some data-preparation and evaluation links are documented rather than shown in excerpts; those edges cite the README.
Main entry points: run_eval.py Evaluation runner · reply_generator.py Reply generator · pipeline.py Agent pipeline · classify_llm.py LLM classifier
Components 19
Agent decisions
- Agent pipeline
pipeline.py - LLM classifier
intents/classify_llm.py - Intent taxonomy
intents/taxonomy.py - Escalation policy
agent/escalation_policy.py - Reply generator
agent/reply_generator.py
Support knowledge
- Similar-case retrieval
retrieval/retrieve.py - Thread reconstruction
data/01_reconstruct_threads.py - Retrieval index builder
retrieval/build_index.py - Resolved-case index
data/processed/Uber_Support_index
Evaluation
- Keyword baseline
intents/classify_baseline_keyword.py - Embedding baseline
intents/classify_embedding_knn.py - Golden examples
eval/golden_set.csv - Evaluation runner
eval/run_eval.py - Reply judge
eval/llm_judge.py - Evaluation metrics
eval/metrics.py
Other components
- Customer
- Groq API
Connections 24
- Customer → Chat UI: sends message
README.md - Chat UI → Flask API: submits chat
README.md - Flask API → Agent pipeline: runs agent
frontend/app.py - Agent pipeline → LLM classifier: classifies message
pipeline.py - Agent pipeline → Escalation policy: decides escalation
pipeline.py - Agent pipeline → Reply generator: drafts reply
pipeline.py - LLM classifier → Intent taxonomy: uses categories
intents/classify_llm.py - Escalation policy → Intent taxonomy: reads rules
agent/escalation_policy.py - Reply generator → Similar-case retrieval: retrieves cases
agent/reply_generator.py - Reply generator → Intent taxonomy: maps help links
agent/reply_generator.py - Similar-case retrieval → Resolved-case index: loads vectors and records
retrieval/retrieve.py - Thread reconstruction → Retrieval index builder: prepares resolved threads
README.md - Retrieval index builder → Resolved-case index: builds index
README.md - LLM classifier → Groq API: requests classification
intents/classify_llm.py - Reply generator → Groq API: requests draft
agent/reply_generator.py - Evaluation runner → Golden examples: reads examples
eval/run_eval.py - Evaluation runner → Keyword baseline: runs baseline
eval/run_eval.py - Evaluation runner → Embedding baseline: runs baseline
eval/run_eval.py - Evaluation runner → LLM classifier: runs classifier
eval/run_eval.py - Evaluation runner → Escalation policy: tests policy
eval/run_eval.py - Evaluation runner → Reply generator: evaluates drafts
eval/run_eval.py - Evaluation runner → Reply judge: scores replies
eval/run_eval.py - Evaluation runner → Evaluation metrics: reports metrics
eval/run_eval.py - Reply judge → Groq API: requests judgment
eval/llm_judge.py