How abhiagni11/tagore-speaks works
Main Components
• Frontend:
• A React application with TypeScript and Tailwind CSS.
• Components include Chat (chat container, chat input, message list, audio visualization), Header, Theme Toggle, and various hooks and services (e.g., authentication with Cartesia API, speech recognition/synthesis).
• Backend:
• A Python Flask server acting as an API layer.
• Organized modules such as:
▪ Routes (e.g., chat_routes in the “routes” directory).
▪ Services (e.g., anthropic_service and response_service) that handle business logic and integration with AI models.
▪ Tools (custom LLM tools like tagore_tools for special features).
• Data Layer:
• Two SQLite databases: one for Tagore’s literary creations (located in tagore-data/tagore-data) and another for conversation storage (as hinted by file names in the data directory).
• External Services:
• Anthropic API for AI conversational responses.
• Cartesia API for voice synthesis (text-to-speech), plus Web Speech API (in the frontend) for speech recognition.
Relationships and Interactions
• The User Interface on the frontend sends chat messages (text or voice) via REST calls to the backend API.
• The backend processes the request:
• Routes in Flask capture the chat requests.
• Depending on the request, services call the Anthropic API (with a specialized system prompt) to generate Tagore-like responses.
• If needed, custom LLM tools are used to fetch or list specific works (like “list_works” or “get_work_content”).
• The backend communicates with the SQLite database(s):
• One database stores the literary content (for retrieving Tagore’s works).
• Another stores the conversation history to be used for persistent memory.
• The response is sent back to the frontend, where additional services may call the Cartesia API to generate speech from text and apply visual/audio feedback.
Architectural Patterns
• Client-Server Pattern: Clearly demarcates the responsibilities between the frontend (client) and backend (server).
• RESTful API Architecture: The backend uses REST endpoints (defined in routes) for communication.
• Separation of Concerns: Distinct folders for routes (API endpoints), services (business logic/communication with external APIs), and tools (custom operations on Tagore’s content).
• Data Flow: User inputs → Frontend → Backend (processing, API calls, DB access) → External Services → Backend response → Frontend display and voice synthesis.
Final Checklist
• Clear labels for every component and sub-component.
• That custom LLM tools and conversation memory (persistent storage) aspects are included since they are critical to how the system personalizes and tracks conversations.