How akshatv07/email-automation works
Project type and purpose
• This is a backend-only application (a CLI/service) for automating ticket resolution via semantic search. It ingests CSV and Excel knowledge-base data, indexes it in a Milvus vector store, and then uses a semantic response engine to generate email replies based on incoming tickets.
File structure
• Top-level scripts and entry points
• main.py – orchestrates end-to-end flow
• bedrock.py – integration with AWS Bedrock (optional LLM provider)
• email_responder.py, retrieve_email.py, manual_email_processor.py – handle email I/O and manual overrides
• utility scripts (list_collections.py, inspect_collections.py, delete_all_collections.py) – Milvus collection management
• config/
• settings.py – global configuration (Milvus host/port, email server, file paths)
• data/
• datadb.csv – ticket metadata
• kb_sheets/*.xlsx – raw knowledge-base content
• core/
• data_db_processor.py – transforms CSV metadata into structured form
• vector_ingestion_engine.py – converts KB into vector embeddings and pushes to Milvus
• (semantic_response_engine.py per README) – vector query + response extraction
• templates/templates/
• dozens of HTML templates keyed by ticket category for email responses
• tests and examples
• test_ingest.py, test_robust_query.py – unit tests
• example_integration.py – sample usage
Key insights from README
• Milvus runs in Docker via docker-compose.yml
• Python 3.8+, venv, pip-installable requirements.txt
• SemanticResponseEngine is the core API for querying
• End-to-end: ingest → semantic store → query → respond
Main components
a. Data Sources
• CSV metadata (data/datadb.csv)
• Excel KB sheets (data/kb_sheets/*.xlsx)
b. Data Processing & Ingestion
• DataDB Processor (core/data_db_processor.py)
• Vector Ingestion Engine (core/vector_ingestion_engine.py)
c. Storage & Index
• Milvus Vector Store (Docker container)
• Metadata store (could be local files or relational DB)
d. Semantic Response Engine
• Semantic query module (core/semantic_response_engine.py)
• Embedding model (OpenAI/AWS Bedrock via bedrock.py)
e. Email Handling
• Retrieve incoming emails (retrieve_email.py)
• Route tickets to semantic engine
• Format responses via Jinja-style HTML templates
• Send replies (email_responder.py)
• Manual override tool (manual_email_processor.py)
f. Utilities & Admin
• Collection management scripts
• Configuration module (config/settings.py)
Relationships and interactions
• Ingestion pipeline: raw files → DataDB Processor → Vector Ingestion → Milvus
• Query flow: incoming email → SemanticResponseEngine → template lookup → Email Responder → SMTP
• Admin tools talk directly to Milvus for maintenance
Architectural patterns and design principles
• Separation of concerns: ingestion vs query vs email I/O
• Pipeline architecture for data flow
• Single responsibility modules for each stage
• Extensibility: new embedding models (bedrock.py), new templates
Technologies and frameworks
• Python 3.8+
• Docker / Milvus (vector database)
• Pandas for CSV/XLSX processing
• Jinja-style HTML templating
• SMTP/IMAP libraries for email I/O
• AWS Bedrock or OpenAI embeddings