# 2977094657/bilibilihistoryfetcher architecture

> Architecture diagram of the GitHub repository 2977094657/bilibilihistoryfetcher, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/2977094657/bilibilihistoryfetcher
- Repository: https://github.com/2977094657/bilibilihistoryfetcher

## Overview

##### Project Identification
- Purpose: This is a backend system that fetches, processes, analyzes, and visualizes a Bilibili user’s viewing history. It supports data collection, various analytical reports (annual summaries, video, and image downloads), automation (scheduler tasks), and even audio-to-text via GPU-accelerated modules.
- Type: A data-driven backend service that exposes API endpoints (via the “routers” directory) and interacts with utility scripts. It is coupled with a separate front-end project (BiliHistoryFrontend) but primarily focuses on API and automation tasks.

##### Analyzing the File Structure
- Configuration and Setup:
- Files in the “config” directory (config.yaml, scheduler_config.yaml, etc.) indicate centralized configuration for B站 API authentication, email notifications, DeepSeek API, server settings, and scheduler tasks.
- Files like CUDA_SETUP.md and install_dependencies.py confirm dependency and hardware configuration (CUDA/CPU variants, PyTorch, etc.).
- Core Application:
- “main.py” is the entry point, likely setting up the server and initializing routes.
- “app.spec”, “app_full.spec”, “app_lite.spec”, and “build.py” are used for packaging the app into standalone executables, with different build options (Lite vs Full versions).
- API Routing:
- The “routers” folder contains individual modules for various endpoints (e.g., analysis, audio_to_text, login, download, scheduler, etc.). Each of these corresponds to a specific functionality exposed via an HTTP API.
- Utility & Data Processing:
- The “scripts” directory hosts multiple scripts handling tasks like data extraction, cleaning, analysis (e.g., analyze_bilibili_history.py, comment_fetcher.py, scheduler-related scripts), which can either be invoked manually or scheduled.
- Dependency & Environment Management:
- Files like requirements.txt and install_dependencies.py ensure that all required Python packages are installed along with special handling for GPU acceleration.

##### Key Components
a. Main Components:
- Client / Front-end: (External) The BiliHistoryFrontend that consumes the API.
- API Server: The core HTTP server launched via “main.py” that binds all API endpoints.
- Routers/Endpoints: Each module in “routers” (for fetching history, exporting data, audio-to-text conversion, etc.) should be depicted as separate service nodes or grouped based on functionality.
- Task Scheduler: Represented by both the scheduler endpoints in “routers/scheduler.py” and the scripts managing scheduled tasks (in “scripts/scheduler*.py”).
- Database Access: Although not a full-fledged DB component in itself, include “sql_statements_mysql.py” and “sql_statements_sqlite.py” to indicate interactions with the chosen database (SQLite or MySQL).
- External Integrations:
- Email Service for notifications (configures SMTP settings).
- DeepSeek AI API for advanced summary/analysis.
- FFmpeg for video processing.
- PyTorch and faster-whisper for audio to text conversion.
b. Relationships & Interactions:
- Data flows from user requests (front-end or automated scheduler) to API endpoints, which then process or delegate tasks to the appropriate data processing modules.
- Configuration files provide settings that are read by almost every module (API server, scheduler, and dependent processing scripts).
- The packaging/build process (build.py, app_xxx.spec) wraps the entire functionality into executable bundles.
- Request/Response cycles between Frontend and API Server.
- Data flow from API endpoints to utility scripts/modules for processing.
- Scheduler invocations triggering individual processing tasks and subsequently sending email notifications or visual outputs.
c. Architectural Patterns/Design Principles:
- Separation of concerns: Clearly dividing API routing (routers), configuration (config files), scheduled/background tasks (scripts), and packaging.
- Modularity: Organizing each endpoint or task in its individual module/script enables easy testing and maintenance.
- Extensibility: The design supports additional external integrations (DeepSeek, FFmpeg, CUDA acceleration) without major structural changes, as seen in the configuration and dedicated integration files.
d. Relevant Technologies & Frameworks:
- Python 3.10+, FastAPI (implied by the auto-generated docs at /docs and the modular routers).
- SQLite/MySQL (as possible persistence layers).
- PyInstaller for application packaging.
- CUDA/PyTorch/faster-whisper for audio processing.
- Email (SMTP) configuration for notifications.

##### Specific Elements
- “API Server (main.py)”, “Routers (analysis, login, etc.)”, “Scheduler”, “Data Processing Scripts”, “Config Manager”, “Packaging Module”
- Data flow from Front-end → API Server → Routers → Scripts/External Services.
- Scheduler triggering periodic tasks that feed into data processing flows and then to email notifications.
- Details such as supported technologies (e.g., CUDA-enabled modules, PyTorch dependency) and the build process (Lite vs Full executables).

## Diagram (Mermaid)

```mermaid
flowchart TD
    FE["Frontend (BiliHistoryFrontend)"]:::external
    API_Server["API Server (main.py)"]:::server
    Routers["API Routers (routers)"]:::routers
    Config_Manager["Configuration Manager (config)"]:::config
    Sched_Endpoint["Scheduler Endpoint (routers/scheduler.py)"]:::scheduler
    Sched_Scripts["Scheduler Scripts (scripts)"]:::scheduler
    Data_Processing["Data Processing Modules (scripts)"]:::processing
    Packaging_Build["Packaging/Build Process (build/spec)"]:::build
    Dependencies["Dependencies & Environment (install_dependencies.py)"]:::dependency
    Email_Service["Email Service (SMTP)"]:::external
    DeepSeek["DeepSeek AI API"]:::external
    FFmpeg["FFmpeg"]:::external
    Audio_Conversion["Audio Conversion (PyTorch,Faster-whisper)"]:::external
    Database["Database Access (SQLite/MySQL)"]:::database

    FE -->|"sends request"| API_Server
    API_Server -->|"routes to"| Routers
    API_Server -->|"loads"| Config_Manager
    Routers -->|"handles scheduler"| Sched_Endpoint
    Routers -->|"triggers data processing"| Data_Processing
    Config_Manager -->|"configures"| Sched_Endpoint
    Config_Manager -->|"provides settings for"| Data_Processing
    Sched_Endpoint -->|"invokes tasks"| Sched_Scripts
    Sched_Scripts -->|"launches data jobs"| Data_Processing
    Data_Processing -->|"sends notifications to"| Email_Service
    Data_Processing -->|"queries"| DeepSeek
    Data_Processing -->|"calls"| FFmpeg
    Data_Processing -->|"utilizes"| Audio_Conversion
    Data_Processing -->|"stores results in"| Database
    Packaging_Build -->|"packages"| API_Server
    Dependencies -->|"installs for"| API_Server

    click API_Server "https://github.com/2977094657/bilibilihistoryfetcher/blob/master/main.py"
    click Routers "https://github.com/2977094657/bilibilihistoryfetcher/tree/master/routers/"
    click Config_Manager "https://github.com/2977094657/bilibilihistoryfetcher/tree/master/config/"
    click Sched_Endpoint "https://github.com/2977094657/bilibilihistoryfetcher/blob/master/routers/scheduler.py"
    click Sched_Scripts "https://github.com/2977094657/bilibilihistoryfetcher/blob/master/scripts/scheduler.py"
    click Data_Processing "https://github.com/2977094657/bilibilihistoryfetcher/blob/master/scripts/analyze_bilibili_history.py"
    click Packaging_Build "https://github.com/2977094657/bilibilihistoryfetcher/blob/master/build.py"
    click Dependencies "https://github.com/2977094657/bilibilihistoryfetcher/blob/master/install_dependencies.py"

    classDef external fill:#F9E79F,stroke:#933,stroke-width:2px;
    classDef server fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef routers fill:#A9DFBF,stroke:#117A65,stroke-width:2px;
    classDef config fill:#F7DC6F,stroke:#B7950B,stroke-width:2px;
    classDef scheduler fill:#D5F5E3,stroke:#117864,stroke-width:2px;
    classDef processing fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef build fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef dependency fill:#A3E4D7,stroke:#117864,stroke-width:2px;
    classDef database fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
```
