How emocareai/chatpsychiatrist works
Project type and purpose
• ChatPsychiatrist is a domain-specific extension of the FastChat framework: a distributed LLM-based chatbot system fine-tuned on counseling dialogs for mental health support.
• It encompasses data processing, model training, serving (CLI + Web UI), evaluation/monitoring, and integration with external services (GPT-4 API, Huggingface Hub, Nginx).
File structure
• Top-level directories:
• assets, docker, docs – static assets, containerization, documentation
• fastchat – core modules (data, model adaption, training, serving, evaluation, protocol)
• llm-judge – legacy or parallel judging suite
• playground – experimental embedding tests
• scripts, tests – automation and CI tests
• fastchat sub-modules reveal architectural layers:
• data/… – data cleaning, conversion, dataset prep (Psych8K)
• model/… – model adapters, LORA/delta application, quantization, monkey patches
• train/… – training entrypoints and strategy scripts (Lora, FlashAttention, etc.)
• serve/… – controller, model_worker, multi_model_worker, gradio web UIs, API provider, gateway config, monitoring scripts
• llm_judge & eval/… – generation of model answers, judgments, scoring, table generation
• protocol/… – definition of API protocols (OpenAI style)
README insights
• Pipeline: raw counseling transcripts → clean & extract → GPT-4 to generate Q-A pairs → fine-tune LLaMA-7B → serve via CLI & Gradio UI → evaluate on counseling bench using custom metrics.
• Core technologies: Python, PyTorch, Huggingface transformers, FastChat, GPT-4 API, Gradio, Docker, Nginx, Huggingface Hub.
a. Main components
• Data Pipeline
• Model Training & Adaptation
• Model Registry/Hub (Huggingface)
• Serving Layer
– Controller (coordination)
– Worker(s) (model inference)
– API Provider & Gateway (nginx)
– Gradio Web UI & CLI
• Evaluation & Monitoring
• External Services
– OpenAI GPT-4 API (dataset creation, judgment)
– Huggingface Hub (model download/upload)
b. Relationships / data flow
1. Data Pipeline outputs Psych8K dataset → Training module
2. Training produces model artifacts → optionally compress/quantize → upload to Huggingface Hub
3. Serving layer pulls model artifacts from Hub → Controller schedules inference requests to Workers
4. User interacts via CLI or Web UI → requests go through API Provider → routed by Controller to Worker → responses return to UI/CLI
5. Evaluation module invokes model (via CLI API) on bench questions → uses GPT-4 API to judge outputs → generates metrics and tables
6. Monitoring scripts collect logs/metrics from serve/worker processes → dashboard or CSV/HTML
c. Architectural patterns & principles
• Modular layered architecture: clear separation of data, model, training, serve, evaluation
• Distributed worker pattern: Controller + Worker processes for horizontal scaling
• Plugin/adaptor pattern: protocol adapters (OpenAI, API), model adapters (LORA, Delta)
• CI/CD and containerization: Docker + docker-compose for reproducible environments
d. Relevant technologies & frameworks
• FastChat framework (core orchestration)
• Transformers & PEFT (LORA, delta) for fine-tuning
• vLLM, FlashAttention, xFormers for performance
• Gradio for Web UI, CLI for terminal usage
• Nginx for HTTP gateway
• Docker for environment isolation
• OpenAI GPT-4 API for data generation and judging
• Huggingface Hub for model hosting