How asavinov/intelligent-trading-bot works
Project type and purpose
• This is a Python‑based, data‑pipeline + ML‑driven trading bot. It consists of two major “subsystems”:
a. An offline batch pipeline (scripts under scripts/, backed by utility code in common/) that downloads market data, merges it, generates features and labels, trains ML models, aggregates scores, and optimizes signal parameters.
b. An online service (code under service/, with connectors in inputs/ and outputs/) that periodically pulls live data, computes features, applies pre‑trained models, aggregates scores, issues buy/sell signals, and optionally places real trades and sends Telegram notifications.
File‑structure analysis
• Top‑level directories:
• common/ – shared routines: backtesting, feature/label generators, signal aggregation, model‑store definitions.
• scripts/ – offline batch steps (download_…, merge, features, labels, train, signals, train_signals, plus helpers for Yahoo/rolling prediction).
• inputs/ – live data collectors (e.g., collector_binance.py).
• outputs/ – notification/trading adapters (e.g., notifier_*.py, trader_binance.py).
• service/ – the server.py entrypoint, App.py config, real‑time analyzer.
• configs/ – sample JSONC config files.
• docs/, tests/, requirements.txt, etc.
Key README insights
• Offline steps produce a model bundle in MODELS/.
• Online service (python -m service.server -c config.json) requires those models.
• External integrations: Binance REST/WebSocket API, Telegram Bot API.
• ML frameworks: scikit‑learn (via model_store.py), TA‑lib, tsfresh, custom algorithms.
Main components
A. Offline Batch Pipeline (left side)
1. Data Sources
– Binance API (scripts/download_binance)
– Yahoo API (scripts/download_yahoo)
2. Merge & Rasterize (scripts/merge)
3. Feature Generation (scripts/features + common/gen_features)
4. Label Generation (scripts/labels + common/gen_labels_*)
5. Model Training (scripts/train + common/model_store)
6. Score Aggregation (scripts/signals + scripts/train_signals)
7. Artifact Store
– Time‑series CSV/Parquet files (under data_folder)
– Trained model files (under MODELS/)
B. Online Service (right side)
1. Live Data Collector (inputs/collector_binance)
2. In‑Memory Data Window (config‑driven history buffer)
3. Feature Computation (common/gen_features rolling)
4. Prediction Engine (service/analyzer applies models)
5. Score Aggregator (common/gen_signals)
6. Signal Orchestrator (service/server + trade_model rules)
7. Notification Service (outputs/notifier_*.py → Telegram)
8. Trader Service (outputs/trader_binance → Binance execution)
C. Shared Libraries
– common/ utilities used in both offline and online paths.
Relationships and data flows
• Data sources → Merge → Features → Labels → Training → Model Store
• Model Store + Live Data → Prediction Engine → Aggregation → Signals/Trades → External APIs
• feedback loop: best signal thresholds derived offline feed back to online Signal Orchestrator.
Architectural patterns and design principles
• Separation of Concerns: clear offline vs. online pipelines
• Modularity: common/ holds reusable logic; scripts/ form a linear ETL chain; service/ is a long‑running daemon
• Config‑driven: everything—data sources, feature sets, labels, algorithms, thresholds—is defined in JSONC
• Extensibility: new feature generators (in common/), new label generators, new output hooks
Essential details to capture
• The data‑artifact lifecycle from raw download to trained model.
• The shared code in common/ that powers both scripts and service.
• The interaction between Signal Orchestrator and Notification/Trader adapters.
• The external touchpoints: Binance and Telegram.