How tanujkumar24/multi-lingual-ai-assistant works
Project type and purpose
• This is a Python‐based, single‐page web application built with Streamlit.
• It implements a “multilingual assistant” that takes user speech, converts it to text, sends it to a language model (PaLM2), and then renders and speaks back the response via text-to-speech.
File structure
• At the root:
• app.py – the Streamlit entry point that ties UI to backend logic.
• requirements.txt / setup.py / multilingual_assistant.egg-info – packaging + dependencies.
• speech.mp3 – sample media asset.
• research/ – Jupyter notebook for experiments; not part of runtime.
• src/
• helper.py – utility functions for calling Google APIs.
• template.py – UI rendering helpers or response templates.
README
• Tech stack: Python, Streamlit, Google APIs (Speech-to-Text, PaLM2, Text-to-Speech).
• Runs locally via streamlit run app.py or on an EC2 instance.
• Requires a .env file with GOOGLE_API_KEY.
Main components
a. Client / User
• Browser / Streamlit front-end UI
b. Streamlit Server (app.py)
• UI layer (widgets, audio capture, display)
• Controller/business logic (routes user requests to helper functions)
c. Application modules (src/helper.py, src/template.py)
• API integration layer
d. External services (Google Cloud APIs)
• Speech-to-Text (s2t)
• Language Model (PaLM2)
• Text-to-Speech (t2s)
e. Deployment environment (optional)
• AWS EC2 instance running the Streamlit process
Relationships and data flows
• User → Browser UI: clicks “Record” or types text
• Browser → Streamlit Server: HTTP/WebSocket requests carrying audio blobs or text
• Streamlit Server → Helper Module → Google Speech-to-Text API (audio → transcript)
• Google STT API → Helper → Streamlit Server (transcript)
• Streamlit Server → PaLM2 API (prompt + transcript → response text)
• PaLM2 API → Streamlit Server (response text)
• Streamlit Server → Google TTS API (text → speech audio)
• Google TTS → Streamlit Server → Browser (plays audio)
Architectural patterns / design principles
• Layered architecture:
• Presentation (Streamlit UI)
• Orchestration / Controller (app.py)
• Integration layer (helper.py calls external APIs)
• Single‐process, single‐codebase design (no microservices)
• Clear separation of concerns: UI vs. business logic vs. external dependencies
Essential elements
• Clear labels on every node (e.g., “Streamlit Server”, “Google TTS”)
• Arrowheads showing direction of data (uni- or bi-directional)