# voidful/tw_stocker architecture

> Architecture diagram of the GitHub repository voidful/tw_stocker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/voidful/tw_stocker
- Repository: https://github.com/voidful/tw_stocker

## Overview

##### Project type and purpose
- This is a data‐driven full‐cycle analytics pipeline for Taiwan stock data.
- Purpose:
- Ingest and store 5-minute interval stock quotes (last 60 days) from Yahoo Finance.
- Compute technical indicators (via `fta`).
- Backtest user‐selectable trading strategies (`strategy/`).
- Generate HTML reports and stock recommendations.
- Automate daily updates and report generation via GitHub Actions.

##### File structure
- Top‐level:
- `.github/workflows/`: two CI/CD pipelines
– `update_data.yml`: fetch new CSVs from Yahoo Finance on schedule
– `update_recommend.yml`: run analysis & regenerate recommendation/report
- `data/`: raw CSV time series files (one per ticker)
- `strategy/`: trading strategy implementations (`dynamic_delay.py`, `grid.py`, etc.)
- `templates/`: HTML templates for final report (`stock_report_template.html`)
- Core scripts:
– `update.py`: orchestrates data ingest & indicator extraction
– `recommend.py`: runs backtest, picks top tickers, populates report
– `example.py`: demo of data fetch & indicator extraction
- Output:
– `stock_report.html`: generated, published report
– `img/`: sample result images
- `requirements.txt`: Python dependencies (`fta`, `vectorbt`, `pandas`, `matplotlib`, etc.)

##### Extract insights from README
- Data comes from Yahoo Finance, updated via GitHub Actions every 5 minutes (?) or daily.
- Technical indicators computed using the `fta` library.
- Backtesting via both custom trade functions and `vectorbt`.
- Reports are static HTML pages published (likely via GitHub Pages).

a. Main components
1. External Service: Yahoo Finance REST API
2. CI/CD Scheduler: GitHub Actions
3. Data Lake: CSV files in `data/` directory (Git repo)
4. Processing Engine: Python scripts (`update.py`, `recommend.py`, `strategy/`)
5. Analytics Libraries: `fta` (TA features), `vectorbt` (portfolio backtest)
6. Report Renderer: Jinja2-style templates (`templates/stock_report_template.html`)
7. Artifact Store & Web Host: GitHub Pages (publishes `stock_report.html`)
8. User Interface: Static HTML report + charts (`img/`, embedded graphs)
9. Client Consumers: Python users importing CSV via raw GitHub URL

b. Relationships & data flows
- GitHub Actions “Update Data” triggers `update.py` → calls Yahoo Finance API → writes/commits CSVs to `data/`.
- GitHub Actions “Update Recommend” triggers `recommend.py`:
– Reads CSVs → uses `fta` to compute indicators → runs strategies in `strategy/` → does backtests → selects top tickers → uses `templates/` to generate `stock_report.html` → commits + pushes.
- GitHub Pages serves `stock_report.html` and raw CSVs under `data/`.
- End users and scripts fetch CSVs via raw GitHub URLs; optionally load into pandas for custom analysis.

c. Architectural patterns & principles
- Pipeline pattern: clear stages (Ingest → Process → Analyze → Report)
- Infrastructure as Code: CI/CD workflows define schedule & build steps.
- Template-based rendering for decoupling logic & presentation.
- Modular strategy plugins: each file in `strategy/` is a pluggable algorithm.

d. Key technologies & frameworks
- Python ecosystem: pandas, matplotlib
- TA computation: `fta`
- Backtesting: `vectorbt`
- CI/CD: GitHub Actions
- Hosting: GitHub Pages (static site)
- Templates: Jinja2 (via built-in Python templating)

##### Detail
- Stage separately: ingestion, indicator computation, strategy execution, backtest, templating, hosting.
- Git repo as both code and artifact store.
- That end users can bypass the pipeline by directly hitting raw CSV URLs.
- Don’t just group everything as “Python app”; break it into scripts, strategy plugins, and template renderer.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Orchestration Layer
    subgraph "Orchestration" 
        direction TB
        GA1["Update Data Workflow"]:::orchestration
        GA2["Update Recommend Workflow"]:::orchestration
    end

    %% External Services
    subgraph "External Services"
        direction TB
        Yahoo["Yahoo Finance API"]:::external
    end

    %% Storage Layer
    subgraph "Data Lake (CSV Store)" 
        direction TB
        DataLake["CSV Files (data/)"]:::storage
    end

    %% Processing Engine
    subgraph "Processing Engine" 
        direction TB
        subgraph "Ingestion & Indicator Extraction"
            direction TB
            UPD["update.py"]:::compute
            EXM["example.py"]:::compute
        end
        subgraph "Recommendation & Backtesting"
            direction TB
            REC["recommend.py"]:::compute
            subgraph "Strategy Plugins"
                direction TB
                SD["dynamic_delay.py"]:::compute
                SG["grid.py"]:::compute
                SJ["just_keep_buying_with_technicals.py"]:::compute
            end
        end
        LIB["requirements.txt"]:::compute
    end

    %% Template Renderer
    subgraph "Report Renderer"
        direction TB
        TMP["stock_report_template.html"]:::compute
    end

    %% Output & Hosting
    subgraph "Artifacts & Hosting"
        direction TB
        RPT["stock_report.html"]:::output
        IMG["img/"]:::output
        GHP["GitHub Pages"]:::orchestration
    end

    %% User Consumers
    subgraph "Clients"
        direction TB
        USR["End Users / Python Scripts"]:::user
    end

    %% Data Flow
    GA1 -->|trigger| UPD
    UPD -->|"fetch quotes"| Yahoo
    UPD -->|"write CSVs"| DataLake
    DataLake -->|"raw CSVs"| UPD

    GA2 -->|trigger| REC
    REC -->|"read CSVs"| DataLake
    REC -->|"compute indicators"| LIB
    REC -->|"load strategies"| SD
    REC -->|"load strategies"| SG
    REC -->|"load strategies"| SJ
    REC -->|"render report"| TMP
    TMP -->|"generate"| RPT
    REC -->|"save images"| IMG
    REC -->|"push report"| GHP

    DataLake -->|"host CSVs"| GHP
    USR -->|"fetch CSVs"| GHP

    %% Click Events
    click GA1 "https://github.com/voidful/tw_stocker/blob/main/.github/workflows/update_data.yml"
    click GA2 "https://github.com/voidful/tw_stocker/blob/main/.github/workflows/update_recommend.yml"
    click DataLake "https://github.com/voidful/tw_stocker/tree/main/data/"
    click UPD "https://github.com/voidful/tw_stocker/blob/main/update.py"
    click EXM "https://github.com/voidful/tw_stocker/blob/main/example.py"
    click REC "https://github.com/voidful/tw_stocker/blob/main/recommend.py"
    click SD "https://github.com/voidful/tw_stocker/blob/main/strategy/dynamic_delay.py"
    click SG "https://github.com/voidful/tw_stocker/blob/main/strategy/grid.py"
    click SJ "https://github.com/voidful/tw_stocker/blob/main/strategy/just_keep_buying_with_technicals.py"
    click LIB "https://github.com/voidful/tw_stocker/blob/main/requirements.txt"
    click TMP "https://github.com/voidful/tw_stocker/blob/main/templates/stock_report_template.html"
    click RPT "https://github.com/voidful/tw_stocker/blob/main/stock_report.html"
    click IMG "https://github.com/voidful/tw_stocker/tree/main/img/"
    click USR "https://github.com/voidful/tw_stocker/blob/main/README.md"

    %% Styles
    classDef external fill:#bbdefb,stroke:#1976d2,color:#0d47a1;
    classDef orchestration fill:#e1bee7,stroke:#7b1fa2,color:#4a148c;
    classDef storage fill:#ffe0b2,stroke:#fb8c00,color:#e65100;
    classDef compute fill:#c8e6c9,stroke:#388e3c,color:#1b5e20;
    classDef output fill:#b2dfdb,stroke:#00796b,color:#004d40;
    classDef user fill:#d7ccc8,stroke:#5d4037,color:#3e2723;
```
