# pythonstock/stock architecture

> Architecture diagram of the GitHub repository pythonstock/stock, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pythonstock/stock
- Repository: https://github.com/pythonstock/stock

## Overview

##### Project Context and Purpose
- This is a full‐stack stock analysis system developed in Python. Its primary purpose is to fetch stock data (using libraries like akshare), perform data cleaning/processing (with pandas, numpy, etc.), compute various technical indicators, and then serve the results via a Tornado-based web API. The frontend is built with Vue and supports data visualization (using bokeh) and interactive report panels.
- The project also uses scheduled cron jobs for periodic data fetching and processing, and it leverages Docker (via docker-compose) for both development and production deployments.

##### High-Level Components
a. Backend Components:
- Jobs/Task Layer: Under the “backend/jobs” directory are multiple Python scripts (daily, hourly, minutely, etc.) responsible for data grabbing, indicator computations, and scheduled updates.
- Libraries/Utilities: “backend/libs” contains common utility functions and dictionaries (for stock web lookup) that support various backend processes.
- Web Server/API: “backend/web” is dedicated to the Tornado-based web application. This layer handles HTTP requests, renders templates, and embeds bokeh plots.
- Process/Service Management: “backend/supervisor” – files for managing long-running processes indicate use of a supervisor (or similar process manager) to ensure the continuous running of jobs.

b. Frontend Components:
- The “frontend” folder is a Vue-based project. It includes source files for Vue components, layouts, routing, API connectors to the backend, unit tests, and configuration files. This part of the system is responsible for presenting interactive charts, reports, and dashboards to end users.

c. Database/Data Storage:
- The README specifies that stock data is stored in a MySQL (or MariaDB but ultimately MySQL is used) database, deployed and initialized through provided SQL scripts (e.g., in the docker-compose/mysql folder).

d. Containerization & Deployment:
- The “docker-compose” folder contains Dockerfiles for production and development environments, along with compose YAML files. These files deploy the backend, frontend, and MySQL containers. They also include additional scripts to build images and manage container entrypoints.

##### Relationships and Data Flows
- Data Flow from External Sources → Data Ingestion:
- Scheduled cron jobs (in “backend/jobs”) grab stock data via akshare. These jobs run at various intervals (minutely, hourly, daily, etc.) and write processed data both to disk (cache files using pickle with gzip compression) and to the MySQL database.

- Data Processing:
- After ingestion, various processing scripts (both new jobs and legacy ones in “backend/old_jobs”) calculate technical indicators (KDJ, RSI, MACD, etc.) based on the fetched data.

- API and Data Exposure:
- The Tornado web server (in “backend/web”) serves as an API endpoint for both the frontend and external clients. It handles client requests, queries computation results (from MySQL or cached files), and renders web pages (templated HTML pages and embedded bokeh charts).

- Frontend Interaction:
- The Vue-based frontend (in “frontend”) consumes the backend API endpoints. It displays dashboards, interactive charts, and tabular reports. The frontend and backend communicate via HTTP (with endpoints clearly delineated in configuration files and documented in the README).

- Containers & Deployment:
- Through docker-compose, the entire stack is containerized. A typical deployment includes separate containers for the web server (backend), frontend application, and MySQL database. The docker-compose setup also provides isolation of environments (production versus development).

##### Architectural Patterns & Key Technologies
- Separation of Concerns: The project is neatly divided based on functionality:
- Jobs (data ingestion/processing) are decoupled from the web API.
- The web UI (frontend) is separated from core Python processing logic (backend).
- Use of Scheduled Tasks: Cron jobs manage the periodic data processing tasks.
- Docker & Containerization: The use of docker-compose allows a unified and portable deployment model.
- Technology Stack: Python (Tornado, pandas, numpy, akshare, bokeh), Vue.js, MySQL, and Docker.

- Use clear labels for each major component (e.g., “Data Ingestion (cron jobs)”, “Backend Processing & API (Tornado)”, “Frontend (Vue.js)”, “Database (MySQL)”).
- External dependencies (libraries like pandas, akshare, bokeh) where relevant, noting that these provide key functionalities within their respective modules.

## Diagram (Mermaid)

```mermaid
flowchart TD
    External["External Stock Data Providers"]:::external
    deployment["Deployment (Docker Compose)"]:::container

    subgraph "Backend Services"
        jobs["Data Ingestion (Cron Jobs)"]:::backend
        legacy["Legacy Jobs"]:::backend
        libs["Libraries/Utilities"]:::backend
        supervisor["Process Manager (Supervisor)"]:::backend
        tornado["Backend API/Web Server (Tornado)"]:::backend
    end

    subgraph "Database Services"
        mysql["MySQL Database"]:::database
    end

    subgraph "Frontend Services"
        frontend["Frontend Application (Vue)"]:::frontend
    end

    deployment --> jobs
    deployment --> mysql
    deployment --> frontend

    External -->|"fetchesData"| jobs
    jobs -->|"writesData"| mysql
    jobs -->|"processesData"| tornado
    legacy --> tornado
    supervisor --> jobs
    tornado -->|"dataOps"| mysql
    frontend -->|"HTTPcalls"| tornado

    click jobs "https://github.com/pythonstock/stock/tree/master/backend/jobs"
    click legacy "https://github.com/pythonstock/stock/tree/master/backend/old_jobs"
    click libs "https://github.com/pythonstock/stock/tree/master/backend/libs"
    click supervisor "https://github.com/pythonstock/stock/tree/master/backend/supervisor"
    click tornado "https://github.com/pythonstock/stock/tree/master/backend/web"
    click frontend "https://github.com/pythonstock/stock/tree/master/frontend"
    click mysql "https://github.com/pythonstock/stock/tree/master/docker-compose/mysql"
    click deployment "https://github.com/pythonstock/stock/tree/master/docker-compose"

    classDef external fill:#f9c,stroke:#333,stroke-width:2px;
    classDef container fill:#ffe4b5,stroke:#8b4513,stroke-width:2px;
    classDef backend fill:#add8e6,stroke:#00008b,stroke-width:2px;
    classDef database fill:#90ee90,stroke:#006400,stroke-width:2px;
    classDef frontend fill:#ffe4e1,stroke:#ff69b4,stroke-width:2px;
```
