How voidful/tw_stocker works
Project type and purpose
• This is a data‐driven full‐cycle analytics pipeline for Taiwan stock data.
• Purpose:
• Ingest and store 5-minute interval stock quotes (last 60 days) from Yahoo Finance.
• Compute technical indicators (via fta).
• Backtest user‐selectable trading strategies (strategy/).
• Generate HTML reports and stock recommendations.
• Automate daily updates and report generation via GitHub Actions.
File structure
• Top‐level:
• .github/workflows/: two CI/CD pipelines
– update_data.yml: fetch new CSVs from Yahoo Finance on schedule
– update_recommend.yml: run analysis & regenerate recommendation/report
• data/: raw CSV time series files (one per ticker)
• strategy/: trading strategy implementations (dynamic_delay.py, grid.py, etc.)
• templates/: HTML templates for final report (stock_report_template.html)
• Core scripts:
– update.py: orchestrates data ingest & indicator extraction
– recommend.py: runs backtest, picks top tickers, populates report
– example.py: demo of data fetch & indicator extraction
• Output:
– stock_report.html: generated, published report
– img/: sample result images
• requirements.txt: Python dependencies (fta, vectorbt, pandas, matplotlib, etc.)
Extract insights from README
• Data comes from Yahoo Finance, updated via GitHub Actions every 5 minutes (?) or daily.
• Technical indicators computed using the fta library.
• Backtesting via both custom trade functions and vectorbt.
• Reports are static HTML pages published (likely via GitHub Pages).
a. Main components
1. External Service: Yahoo Finance REST API
2. CI/CD Scheduler: GitHub Actions
3. Data Lake: CSV files in data/ directory (Git repo)
4. Processing Engine: Python scripts (update.py, recommend.py, strategy/)
5. Analytics Libraries: fta (TA features), vectorbt (portfolio backtest)
6. Report Renderer: Jinja2-style templates (templates/stock_report_template.html)
7. Artifact Store & Web Host: GitHub Pages (publishes stock_report.html)
8. User Interface: Static HTML report + charts (img/, embedded graphs)
9. Client Consumers: Python users importing CSV via raw GitHub URL
b. Relationships & data flows
• GitHub Actions “Update Data” triggers update.py → calls Yahoo Finance API → writes/commits CSVs to data/.
• GitHub Actions “Update Recommend” triggers recommend.py:
– Reads CSVs → uses fta to compute indicators → runs strategies in strategy/ → does backtests → selects top tickers → uses templates/ to generate stock_report.html → commits + pushes.
• GitHub Pages serves stock_report.html and raw CSVs under data/.
• End users and scripts fetch CSVs via raw GitHub URLs; optionally load into pandas for custom analysis.
c. Architectural patterns & principles
• Pipeline pattern: clear stages (Ingest → Process → Analyze → Report)
• Infrastructure as Code: CI/CD workflows define schedule & build steps.
• Template-based rendering for decoupling logic & presentation.
• Modular strategy plugins: each file in strategy/ is a pluggable algorithm.
d. Key technologies & frameworks
• Python ecosystem: pandas, matplotlib
• TA computation: fta
• Backtesting: vectorbt
• CI/CD: GitHub Actions
• Hosting: GitHub Pages (static site)
• Templates: Jinja2 (via built-in Python templating)
Detail
• Stage separately: ingestion, indicator computation, strategy execution, backtest, templating, hosting.
• Git repo as both code and artifact store.
• That end users can bypass the pipeline by directly hitting raw CSV URLs.
• Don’t just group everything as “Python app”; break it into scripts, strategy plugins, and template renderer.