akisionov/reinforcement-learning-model-for-stock-market-data

Scope and topology

This is a Python deep-reinforcement-learning trading project split into a model-development package and a paper-trading deployment package. The training side produces PPO model artifacts from engineered DJIA market data; the deployment side loads the model, obtains current market data, computes target weights, and paper-executes or simulates trades.

Key roots: Optimized_RL_Trading_Strategy/, RL_Paper_Trading_Deployment/, README.md.

Training and optimization entry points

Training is orchestrated as runnable Python/notebook workflows. train_optimized.py is the primary model-training entry point; evaluate.py evaluates trained policies; main.ipynb is an interactive workflow, with build_notebook.py supporting notebook construction.

Paths: Optimized_RL_Trading_Strategy/train_optimized.py, Optimized_RL_Trading_Strategy/evaluate.py, Optimized_RL_Trading_Strategy/main.ipynb.

Market-data and feature boundary

The offline pipeline consumes and prepares market dynamics for RL. The persisted engineered dataset contains DJIA assets and features such as HMM regimes, GARCH volatility, and Corwin-Schultz spreads. This dataset is the explicit contract between feature engineering and environment/training consumers.

Paths: Optimized_RL_Trading_Strategy/data_pipeline.py, Optimized_RL_Trading_Strategy/data/processed_market_dynamics.csv.

RL environment and learning contract

The custom environment is the central simulation boundary: it exposes a 567-dimensional observation space and encodes regime-based position clipping, circuit breakers, turnover penalties, transaction costs, and downside-risk-oriented rewards. PPO training interacts with this environment rather than directly with external brokers or live feeds.

Paths: Optimized_RL_Trading_Strategy/custom_env.py, Optimized_RL_Trading_Strategy/train_optimized.py.

Model artifacts and validation

Trained Stable-Baselines/PPO artifacts are committed and consumed across phases. Training produces final/best models and evaluation arrays; stress tooling provides a separate resilience-validation path. Deployment carries its own model copy, making artifact version alignment a material operational boundary.

Paths: Optimized_RL_Trading_Strategy/trained_models/final_model.zip, Optimized_RL_Trading_Strategy/trained_models/evaluations.npz, Optimized_RL_Trading_Strategy/stress_harness_m2.py, RL_Paper_Trading_Deployment/optimal_trading_model.zip.

Live inference and execution

trade_executor.py bridges offline policy output to current trading operations: it fetches real-time Yahoo Finance data, runs model inference for portfolio target weights, and sends paper orders to Alpaca when credentials exist. Without Alpaca keys, it falls back to local mock execution. This is the critical boundary between model decisions and external side effects.

Paths: RL_Paper_Trading_Deployment/trade_executor.py, RL_Paper_Trading_Deployment/.env.example, RL_Paper_Trading_Deployment/secrets_guide.md.

Operator interface and runtime infrastructure

The deployment is a local Streamlit application, launched through dashboard.py, for live portfolio performance, daily P&L, and market-regime visibility. Material runtime dependencies include Python, Stable-Baselines3, Yahoo Finance (yfinance), Streamlit, and Alpaca’s trading API. A scheduled GitHub Actions workflow is present for daily trading automation.

Paths: RL_Paper_Trading_Deployment/dashboard.py, RL_Paper_Trading_Deployment/requirements.txt, RL_Paper_Trading_Deployment/.streamlit/secrets.toml.example, .github/workflows/daily_trading.yml.