# lixi5338619/weixin-spider architecture

> Architecture diagram of the GitHub repository lixi5338619/weixin-spider, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lixi5338619/weixin-spider
- Repository: https://github.com/lixi5338619/weixin-spider

## Overview

##### Main System Components

a) Client Layer:
- Windows WeChat Client
- Web Interface (Flask-based, port 5000)
- File Helper Window (for automation)

b) Proxy Layer:
- Mitmproxy (port 8080)
- Proxy Configuration Component

c) Core Processing Layer:
- Monitor System
- Crawler API
- Task Scheduler

d) Data Storage Layer:
- MySQL Database (weixin_spider)
- Redis Cache

##### Component Relationships and Data Flow

**[WeChat Client] → [Mitmproxy] → [Crawler API]**
- ↓
**[Web Interface] → [Task Scheduler] → [Monitor System]**
- ↓
- [Data Storage Layer]

##### Specific Component Details

For each major component, include:

Web Interface (webapp/):
- Connection to templates
- Static assets
- View controllers

Proxy Layer:
- Parameter extraction (x-wechat-key, __biz, appmsg_token, pass_ticket)
- Connection to automation system

Core Processing:
- Article data processing
- Account information processing
- Task management

##### Key Integration Points

Highlight:
- Mitmproxy integration with WeChat client
- Automation system interaction with File Helper
- API endpoints for data collection
- Database connections

##### Technical Details

Annotations for:
- Port numbers (5000, 8080)
- Database schema reference
- Redis key patterns
- API endpoints

##### Architectural Patterns

Emphasize:
- MVC pattern in webapp
- Event-driven architecture for monitoring
- Proxy pattern for data interception

##### Top-level organization
**[Client Layer]**
- ↓
**[Proxy Layer]**
- ↓
**[Processing Layer]**
- ↓
**[Storage Layer]**

##### Notes for
- Daily collection limits (5000-8000 articles)
- Critical configuration requirements
- Error handling paths

This system design should clearly show the flow from WeChat client through the proxy, to data processing and storage, while highlighting the automation and monitoring aspects that make this system unique.

Remember to maintain a balance between detail and clarity, focusing on the core architectural components while providing enough detail to understand the system's operation.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles
    classDef client fill:#3498db,stroke:#2980b9,color:white
    classDef proxy fill:#f1c40f,stroke:#f39c12,color:black
    classDef processing fill:#2ecc71,stroke:#27ae60,color:white
    classDef storage fill:#e74c3c,stroke:#c0392b,color:white
    classDef automation fill:#9b59b6,stroke:#8e44ad,color:white

    %% Client Layer
    subgraph ClientLayer
        WC[WeChat Client]:::client
        WI["Web Interface<br/>(Port 5000)"]:::client
        FH[File Helper Window]:::automation
    end

    %% Proxy Layer
    subgraph ProxyLayer
        MP["Mitmproxy<br/>(Port 8080)"]:::proxy
        PC[Proxy Config]:::proxy
    end

    %% Core Processing Layer
    subgraph ProcessingLayer
        MS[Monitor System]:::processing
        CA[Crawler API]:::processing
        TS[Task Scheduler]:::processing
        RH[Request Handler]:::processing
    end

    %% Data Storage Layer
    subgraph StorageLayer
        DB[("MySQL Database<br/>(weixin_spider)")]:::storage
        RC[("Redis Cache")]:::storage
    end

    %% Relationships
    WC --> MP
    MP --> CA
    WI --> TS
    TS --> MS
    MS --> CA
    CA --> RH
    RH --> DB
    RH --> RC
    FH --> MS
    PC --> MP

    %% Click Events
    click WI "https://github.com/lixi5338619/weixin-spider/tree/master/webapp/"
    click CA "https://github.com/lixi5338619/weixin-spider/blob/master/api/crawlerapi.py"
    click PC "https://github.com/lixi5338619/weixin-spider/blob/master/tools/proxy.py"
    click RH "https://github.com/lixi5338619/weixin-spider/blob/master/tools/handle.py"
    click MS "https://github.com/lixi5338619/weixin-spider/blob/master/wx_monitor.py"

    %% Legend
    subgraph Legend
        C1[Client Component]:::client
        P1[Proxy Component]:::proxy
        PR1[Processing Component]:::processing
        S1[Storage Component]:::storage
        A1[Automation Component]:::automation
    end
```
