How marketcalls/openadvisor works
Project type and purpose
• OpenAdvisor is a full-stack, self-hostable investment platform.
• Backend: Flask REST-style web app (Python) serving HTML templates and JSON endpoints.
• Frontend: Jinja2 templates + Tailwind CSS/Daisy UI + Alpine.js.
• ML pipeline: CatBoost models trained on stock data, scheduled tasks for data download, training, and predictions.
• Database: SQLite via SQLAlchemy.
• External services: Yahoo Finance (via yfinance), Amazon SES (email).
File structure
Top-level components:
• app.py, config.py, extensions.py – initializes Flask app, loads config, registers extensions (DB, mail, scheduler).
• routes/ – Flask Blueprints for auth, dashboard, predictions, charts, backtest, core.
• models.py, forms.py – data layer and form definitions (MVC pattern).
• templates/, static/ – UI layer (HTML, CSS, JS, images).
• data_downloader.py, training_predictions.py – standalone scripts for scheduled tasks (AP Scheduler).
• catboost_info/, Notebook/ – trained model artifacts and experimental notebooks.
• predictions/ – CSV outputs consumed by the app.
• portfolio.csv – seed data for backtesting.
• requirements.txt – dependency manifest.
Key README insights
• AP Scheduler is used to automate daily data downloads and weekly trainings/predictions.
• ML stack: yfinance → raw data → CatBoost training → catboost_info (model + logs) → predictions CSV.
• User flows: register/login → set preferences → view dashboard & charts → run backtest → receive email alerts.
Main components
A. Client layer
– Web Browser (HTML/CSS/JS)
• Alpine.js components
• Tailwind CSS / Daisy UI styling
B. Web Server layer (Flask app)
– Blueprints (auth, core, dashboard, charts, backtest, predictions)
– Controllers (routes/*.py)
– View layer (templates/*.html)
– Forms (forms.py)
– Business logic & services (extensions.py, config.py)
C. Data layer
– SQLite database (SQLAlchemy ORM via models.py)
– CSV file store (predictions/, portfolio.csv)
– Model artifacts (catboost_info/)
D. ML pipeline & scheduler
– AP Scheduler (embedded in Flask or standalone)
• data_downloader.py (daily job)
• training_predictions.py (weekly job)
– yfinance (data source)
– CatBoost (training & prediction engine)
E. External services
– Yahoo Finance API (via yfinance)
– Amazon SES (email notifications)
Relationships & data flows
1. User → Browser → HTTP(S) → Flask app
2. Flask controllers ↔ Database (read/write user & portfolio data)
3. ML scheduler → yfinance → raw data → CatBoost train → model artifacts → predictions CSV
4. Flask job readers → predictions CSV → incorporate into dashboard & charts
5. Email service invoked by Flask (e.g., on registration, alerts) via Amazon SES
Architectural patterns & design principles
• MVC (models.py, forms.py, templates, routes)
• Layered architecture (presentation, application, domain, infrastructure)
• Modularization with Flask Blueprints
• Batch processing & scheduled tasks for ML
• Single-file database (SQLite) for ease of self-hosting
• External integration patterns (data ingestion, email)
Detailed breakdown
• Presentation Layer: Browser → Static (styles.css, charts.js) + Templates → routes core/dashboard
• Application Layer: Flask app.py → extensions (DB, Mail, Scheduler) → route handlers
• Domain Layer: models.py (User, Portfolio, Prediction), forms.py (Auth, Settings)
• ML Subsystem:
– DataDownloader (daily) → yfinance → raw_data.csv
– Trainer (weekly) → CatBoost → model.bin (catboost_info/)
– Predictor → predictions/*.csv
• Infrastructure Layer: SQLite DB file, CSV file store, environment config (.env)
• External Services: Yahoo Finance API, Amazon SES