How xiaohuihui-ue/wechat-on-airflow works
Main Components
• Web Frontend & Authentication: Although not heavily represented in the file tree, the README clearly outlines a “管理页面” (management page) accessed by users via a browser, and an associated authentication service.
• Reverse Proxy / Nginx: The project uses Nginx for managing inbound HTTPS requests and load balancing between public and internal services.
• Airflow Orchestrator: The core of your project is Airflow (as seen in the README and multiple “dags” directories) which schedules and manages workflows.
• Database & Cache: Components include MySQL (for storing key chat records and other persistent data) and Redis (referenced through both database configurations and utility modules).
• Cloud Functions: Several tasks are offloaded to cloud functions that act for tasks like message reporting and data querying.
• AI Agents & External Services: There is integration with external AI services (e.g., Dify for intelligent customer care, a large model API for AI inference) and custom “agent” modules in the “dags/ai_agent” directory.
• WeChat Integrations: Separate workflows exist for personal WeChat functionality, WeChat public account interactions, and additional integrations based on the “wx_dags”, “wx_mp_dags”, and scf directories.
• Auxiliary Services: Other supporting components include Cron DAGs for periodic tasks, proxy watchers, and utility modules that handle specific networking (e.g., proxy tools) or data tasks.
Representing Relationships and Data Flows
• Within the internal network, show how Nginx interacts with:
• The Airflow module – which schedules and executes DAGs that call various agent scripts found under directories such as “dags/ai_agent” and “dags/ai_tennis_dags”.
• Database systems (MySQL and Redis) that store chat logs and state information.
• External services and cloud functions, which further integrate into intelligent response flows (e.g., Dify, large model service).
Architectural Patterns and Design Principles
• Modular Design: The file tree’s segmentation (e.g., distinct directories for “ai_agent”, “ai_tennis_dags”, “cron_dags”, “wx_dags”, etc.) indicates a modular architecture where each folder encapsulates a part of the overall functionality.
• Separation of Concerns: The system clearly separates concerns by using Apache Airflow to coordinate and schedule tasks rather than embedding this logic in a monolithic service.
• Integration of External Services: The design leverages both open-source tools (Airflow, Nginx, Dify) and cloud services (Tencent Cloud functions for intermediate processing).
• The “dags” directory is central; within it, use clusters to represent:
• “ai_agent”: Each Python file (like “brocad_agent_001.py”, “dify_agent_001.py”, “zacks_ai_agent.py”) becomes an individual agent node, detailing the specific AI tasks they support.
• “ai_tennis_dags”: Separate clusters for “action_score_v1” and “action_score_v2” to show versioning differences, each containing script modules and models.
• “wx_dags” & “wx_mp_dags”: Represent these as integration nodes for WeChat client and public account messaging, including the “handlers” and supporting tools.
• The “database” and “dify_conf” directories indicate the infrastructure for configuration and database provisioning.
• Additional directories (such as “scf” and “supabase-sql-storage”) hint at further subsystem integrations with SQL-based dashboards and storage, which should be represented as supportive components connecting to the central workflow engine.