How tristantanjh/techjam2024 works
Project Overview and Components
• Purpose & Project Type:
• This is a full-stack CRM application built to assist sales professionals in real time by transcribing conversations and offering actionable insights. It combines real-time voice transcription, LLM-based context processing, and dynamic action handling (including integrations with systems like Jira and Gmail).
• The project is split between a React-based frontend (in App/client) and a Python/Flask-based backend (in App/server). There are also notebooks and CSV files for prototyping data and testing integrations.
Main Architectural Components
• Frontend (Client-side):
• Built with React (using Vite as the bundler and Tailwind CSS for styling).
• Structured into multiple directories: Components (with UI widgets and “magicui” animated components), CustomComponents, and Pages which further break down into specific functions (ActionsPage, CallerPage, CopilotPage, DashboardPage, etc.).
• Dedicated layout components for presenting the UI (AppLayout, DefaultLayout, NavbarLayout).
• Communication with backend is performed through web sockets (using SocketIO) for real-time data exchange.
• Backend (Server-side):
• A Python Flask server (located in App/server) handling API endpoints.
• Integrates web sockets for real-time communication with the frontend.
• Contains integration scripts and notebooks that show prototypes for:
• CSV-based databases (for customer, product, and outlet data).
• Gmail and Jira integrations (using scripts like sendGmail.py and various Jira-related Python files).
• Implements core business logic to process transcribed text, perform LangChain LLM chaining (invoking OpenAI's gpt-3.5-turbo), and interact with a prepopulated Neo4j database.
• External and Third-Party Services:
• Azure Speech-To-Text: Used for real-time transcription of call conversations.
• OpenAI API: Powered by gpt-3.5-turbo, used for generating responses and contextual insights.
• LangChain Library: Orchestrates multiple LLM chains to process user queries.
• Neo4j Database: Acts as a knowledge graph backing the proof-of-concept FAQ section (using hybrid searches combining graph search with vector similarity search).
• Integrations with Jira and Gmail, where dynamic actions can be submitted (via API calls) once confirmed through the UI.
• Data Components:
• CSV files (found in App/server/csv_db) which act as temporary or prototype databases.
• Notebooks in the repository facilitate testing and prototyping different components (e.g., transcription, scraping, action handling).
Interactions and Data Flow
• Voice Driven AI Assistant Flow:
• The frontend establishes a websocket connection (via SocketIO) to the backend.
• Audio is streamed from the client to the Flask server.
• The server dispatches audio to Azure Speech-To-Text, receives the transcription, and then passes it into a LangChain LLM chain for contextual processing.
• The system then uses the Neo4j database (along with any knowledge graphs) to refine the context and generate insights, including suggestion of follow-up actions.
• Final follow-up actions are routed (optionally, after user confirmation) to external systems like Jira through API calls.
• Intelligent Query and Action Assistant Flow:
• Provides a chatbot-type interface on the frontend where user inputs queries or uploads CSV files.
• The chatbot’s query is routed to backend handlers which use LangChain workflows to determine if the query requires:
• Retrieval from a CSV data store,
• API calls (for Jira, Gmail, or external services), or
• Database queries.
Architectural Patterns & Design Principles
• Layered/MVC Style Separation:
• Frontend is clearly separated from backend logic.
• Within the frontend, there’s a clear division between components, layouts, and pages.
• Microservices-like Integration:
• Although monolithic in codebase, the integration with external services (Azure, Jira, Gmail) behaves like distributed services.
• Real-Time Communication:
• Modular & Extensible Design:
• Components in the frontend (UI vs. magicui animations vs. custom navigation components) are logically separated.
• On the backend, separate directories and files for integrations (Gmail, Jira) indicate plug-and-play style extensibility.