How vardhan2428/customer-support-bot works
This repository is a Twitter-based AmazonHelp support-agent prototype plus the data and evaluation workflows that support it. A customer message enters the CLI and the end-to-end pipeline coordinates intent classification/resolution, precedent retrieval, escalation assessment, and reply generation. Separate data workflows reconstruct conversations, create annotations and silver labels, train the local classifier, and build retrieval examples; frozen-set evaluation measures the resulting agent. The README documents these capabilities, but several data-workflow implementations were not excerpted, so their internal connections are left unasserted.
Main entry points: pipeline.py Agent pipeline · agent.py Intent classifier · reconstruct_amazon_conversations.py Conversation reconstruction · create_annotation_pilot.py Frozen evaluation set
Components 20
Support agent
- Agent CLI
src/agent/agent_cli.py - Agent pipeline
src/agent/pipeline.py - Intent classifier
src/agent/agent.py - LLM resolver
src/agent/llm_resolver.py - Historical retriever
src/agent/retriever.py - Precedent corpus
src/agent/retriever.py - Escalation decision
src/agent/escalation.py - Reply generator
src/agent/generator.py
Data preparation
- Twitter support data
- Conversation reconstruction
src/reconstruct_amazon_conversations.py - Thread analysis
src/reconstruct_amazon_threads.py - Intent annotation
src/create_annotation_pilot.py - Silver labeling
src/build_silver_corpus.py - Classifier training
src/train_silver_intent_classifier.py - Retrieval corpus build
src/build_retrieval_corpus.py
Evaluation
- Frozen evaluation set
src/create_annotation_pilot.py - Agent evaluation
evaluation/evaluate_agent_frozen.py - Retrieval evaluation
evaluation/evaluate_retrieval.py
Other components
- Customer
- Groq service
Connections 20
- Customer → Agent CLI: enters message
src/agent/agent_cli.py - Agent CLI → Agent pipeline: runs pipeline
src/agent/agent_cli.py - Agent pipeline → Intent classifier: prepares intent
src/agent/pipeline.py - Intent classifier → LLM resolver: resolves low confidence
src/agent/agent.py - Intent classifier → Historical retriever: retrieves precedents
src/agent/agent.py - Precedent corpus → Historical retriever: loads and ranks
src/agent/retriever.py - Agent pipeline → Escalation decision: assesses escalation
src/agent/pipeline.py - Agent pipeline → Reply generator: generates reply
src/agent/pipeline.py - LLM resolver → Groq service: requests completion
src/agent/llm_resolver.py - Reply generator → Groq service: requests completion
src/agent/generator.py - Twitter support data → Conversation reconstruction: provides tweets
- Conversation reconstruction → Thread analysis: supports analysis
- Intent annotation → Frozen evaluation set: creates labeled set
src/create_annotation_pilot.py - Silver labeling → Classifier training: supplies labels
- Classifier training → Intent classifier: trains classifier
- Conversation reconstruction → Retrieval corpus build: supports corpus build
- Retrieval corpus build → Precedent corpus: builds examples
- Frozen evaluation set → Agent evaluation: evaluates pipeline
evaluation/evaluate_agent_frozen.py - Frozen evaluation set → Retrieval evaluation: evaluates retrieval
evaluation/evaluate_retrieval.py - Retrieval evaluation → Historical retriever: queries candidates
evaluation/evaluate_retrieval.py