How xs-h/chatbot works
Project Type and Purpose
• This is a full-stack, microservice-based chatbot system (“心语智友”) that integrates three core AI capabilities:
a. ASR (Automatic Speech Recognition) via a Django service in Docker
b. Chat/LLM inference via a Python-based “chat_backend” using Ollama or llama.cpp models
c. TTS (Text-to-Speech) via CosyVoice in the “tts_backend”
• A simple static frontend delivers a web UI for voice input/output and text chat.
High-Level File/Directory Analysis
• Top‐level “frontend”: static HTML/CSS/JS UI
• “backend”: three subfolders:
• asr_backend: Django REST API for speech-to-text, deployable via Docker Compose
• chat_backend: Python modules (chat.py, rag.py) calling an LLM via HuggingFaceEmbeddings & Ollama
• tts_backend: Python scripts wrapping CosyVoice for speech synthesis
• “resource”: local knowledge bases, vector indices, reference audios
• “logs”: service logs for chat and TTS
• Root “start.py”: orchestration script launching Chat and TTS services
• Conda & requirements files, Docker files, and project documentation
Key README Insights
• start.py boots LLM + TTS services; ASR runs separately in Docker
• Frontend JS (chatbot_dynamic.js) calls three HTTP endpoints: /asr, /chat, /tts
• External dependencies: PyTorch, FAISS, ONNX Runtime, Ollama, CosyVoice models
• Resource folder drives RAG (retrieval-augmented generation) and TTS reference audio
a. Main Components
1. User Browser (Web UI)
2. Static Frontend (index.html, chatbot_dynamic.js, styles.css)
3. ASR Service (Django app in Docker)
4. Chat Service (chat_backend Python APIs + Ollama/llama.cpp LLM engine)
5. TTS Service (tts_backend invoking CosyVoice)
6. Model Engines & Data Stores
‑ Ollama/llama.cpp LLM runtime
‑ CosyVoice TTS models
‑ Resource Store (JSON knowledge files, FAISS index)
7. Logs & Monitoring (chat_api.log, tts_api.log)
b. Relationships & Interactions
• Browser → HTTP POST /asr → ASR Service → TranscribeTask → returns text
• Browser → HTTP POST /chat → Chat Service →
• optional RAG lookup against resource/vector DB
• call local Ollama/llama.cpp model → returns reply text
• Browser → HTTP POST /tts → TTS Service → CosyVoice pipeline → returns audio stream
• start.py orchestrates Chat and TTS service processes
• ASR Service deployed independently via Docker Compose
c. Architectural Patterns & Principles
• Microservices: separate containers/services for ASR, Chat, TTS
• RAG: retrieval-augmented generation using FAISS vector store + JSON KB
• Modular Python: Django for ASR, FastAPI or Flask-like for Chat/TTS
• Orchestration script (start.py) to spin up in-process services
d. Technologies & Frameworks
• Django + Docker for ASR
• Python 3.10, FastAPI/Flask style for Chat & TTS
• Ollama or llama.cpp for LLM inference
• CosyVoice (ONNX + PyTorch) for TTS
• FAISS for vector search, HuggingFaceEmbeddings for RAG
• Conda environments, Docker Compose, Git LFS