# akisionov/reinforcement-learning-model-for-stock-market-data architecture

> Architecture diagram of the GitHub repository akisionov/reinforcement-learning-model-for-stock-market-data, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/akisionov/reinforcement-learning-model-for-stock-market-data
- Repository: https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data

## Overview

#### Scope and topology
This is a Python deep-reinforcement-learning trading project split into a model-development package and a paper-trading deployment package. The training side produces PPO model artifacts from engineered DJIA market data; the deployment side loads the model, obtains current market data, computes target weights, and paper-executes or simulates trades.

Key roots: `Optimized_RL_Trading_Strategy/`, `RL_Paper_Trading_Deployment/`, `README.md`.

#### Training and optimization entry points
Training is orchestrated as runnable Python/notebook workflows. `train_optimized.py` is the primary model-training entry point; `evaluate.py` evaluates trained policies; `main.ipynb` is an interactive workflow, with `build_notebook.py` supporting notebook construction.

Paths: `Optimized_RL_Trading_Strategy/train_optimized.py`, `Optimized_RL_Trading_Strategy/evaluate.py`, `Optimized_RL_Trading_Strategy/main.ipynb`.

#### Market-data and feature boundary
The offline pipeline consumes and prepares market dynamics for RL. The persisted engineered dataset contains DJIA assets and features such as HMM regimes, GARCH volatility, and Corwin-Schultz spreads. This dataset is the explicit contract between feature engineering and environment/training consumers.

Paths: `Optimized_RL_Trading_Strategy/data_pipeline.py`, `Optimized_RL_Trading_Strategy/data/processed_market_dynamics.csv`.

#### RL environment and learning contract
The custom environment is the central simulation boundary: it exposes a 567-dimensional observation space and encodes regime-based position clipping, circuit breakers, turnover penalties, transaction costs, and downside-risk-oriented rewards. PPO training interacts with this environment rather than directly with external brokers or live feeds.

Paths: `Optimized_RL_Trading_Strategy/custom_env.py`, `Optimized_RL_Trading_Strategy/train_optimized.py`.

#### Model artifacts and validation
Trained Stable-Baselines/PPO artifacts are committed and consumed across phases. Training produces final/best models and evaluation arrays; stress tooling provides a separate resilience-validation path. Deployment carries its own model copy, making artifact version alignment a material operational boundary.

Paths: `Optimized_RL_Trading_Strategy/trained_models/final_model.zip`, `Optimized_RL_Trading_Strategy/trained_models/evaluations.npz`, `Optimized_RL_Trading_Strategy/stress_harness_m2.py`, `RL_Paper_Trading_Deployment/optimal_trading_model.zip`.

#### Live inference and execution
`trade_executor.py` bridges offline policy output to current trading operations: it fetches real-time Yahoo Finance data, runs model inference for portfolio target weights, and sends paper orders to Alpaca when credentials exist. Without Alpaca keys, it falls back to local mock execution. This is the critical boundary between model decisions and external side effects.

Paths: `RL_Paper_Trading_Deployment/trade_executor.py`, `RL_Paper_Trading_Deployment/.env.example`, `RL_Paper_Trading_Deployment/secrets_guide.md`.

#### Operator interface and runtime infrastructure
The deployment is a local Streamlit application, launched through `dashboard.py`, for live portfolio performance, daily P&L, and market-regime visibility. Material runtime dependencies include Python, Stable-Baselines3, Yahoo Finance (`yfinance`), Streamlit, and Alpaca’s trading API. A scheduled GitHub Actions workflow is present for daily trading automation.

Paths: `RL_Paper_Trading_Deployment/dashboard.py`, `RL_Paper_Trading_Deployment/requirements.txt`, `RL_Paper_Trading_Deployment/.streamlit/secrets.toml.example`, `.github/workflows/daily_trading.yml`.

Main entry points: [`RL_Paper_Trading_Deployment/trade_executor.py`](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/trade_executor.py) (Trade executor), [`Optimized_RL_Trading_Strategy/trained_models/final_model.zip`](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/trained_models/final_model.zip) (Final PPO model), [`Optimized_RL_Trading_Strategy/train_optimized.py`](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/train_optimized.py) (PPO training), [`Optimized_RL_Trading_Strategy/data/processed_market_dynamics.csv`](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/data/processed_market_dynamics.csv) (Engineered market data)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Model Development"]
  node_data_pipeline["Feature pipeline<br/>Python pipeline<br/>[data_pipeline.py]"]
  node_processed_data[("Engineered market data<br/>CSV dataset")]
  node_rl_env["Trading environment<br/>Gym environment<br/>[custom_env.py]"]
  node_train["PPO training<br/>Training entry point<br/>[train_optimized.py]"]
  node_notebook["Interactive workflow<br/>Jupyter notebook<br/>[main.ipynb]"]
  node_final_model["Final PPO model<br/>Model artifact<br/>[final_model.zip]"]
  node_evaluations["Evaluation results<br/>Evaluation artifact<br/>[evaluations.npz]"]
  node_evaluate["Policy evaluation<br/>Evaluation entry point<br/>[evaluate.py]"]
  node_stress["Stress validation<br/>Stress harness"]
end

subgraph group_deployment["Paper Trading Deployment"]
  node_deployment_model["Deployment model copy<br/>Model artifact"]
  node_executor["Trade executor<br/>Inference and execution<br/>[trade_executor.py]"]
  node_dashboard["Operator dashboard<br/>Streamlit app<br/>[dashboard.py]"]
  node_credentials["Credential setup<br/>Secrets documentation<br/>[.env.example]"]
  node_scheduler["Daily trading schedule<br/>GitHub Actions workflow<br/>[daily_trading.yml]"]
  node_mock_execution["Local mock execution<br/>Fallback executor"]
end

subgraph group_external["External Services"]
  node_yahoo{{"Yahoo Finance<br/>Market data API"}}
  node_alpaca{{"Alpaca paper trading<br/>Broker API"}}
end

node_data_pipeline -->|"writes"| node_processed_data
node_processed_data -->|"provides features"| node_rl_env
node_rl_env -->|"simulation interface"| node_train
node_notebook -.->|"runs"| node_train
node_train -->|"produces"| node_final_model
node_train -->|"writes"| node_evaluations
node_final_model -->|"evaluates"| node_evaluate
node_final_model -->|"stress tests"| node_stress
node_final_model -.->|"version-aligned copy"| node_deployment_model
node_deployment_model -->|"loads policy"| node_executor
node_yahoo -->|"current market data"| node_executor
node_credentials -.->|"configures broker access"| node_executor
node_executor -->|"paper orders"| node_alpaca
node_executor -.->|"without credentials"| node_mock_execution
node_dashboard -->|"trading and portfolio view"| node_executor
node_scheduler -->|"runs daily"| node_executor

click node_data_pipeline "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/data_pipeline.py"
click node_processed_data "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/data/processed_market_dynamics.csv"
click node_rl_env "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/custom_env.py"
click node_train "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/train_optimized.py"
click node_notebook "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/main.ipynb"
click node_final_model "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/trained_models/final_model.zip"
click node_evaluations "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/trained_models/evaluations.npz"
click node_evaluate "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/evaluate.py"
click node_stress "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/stress_harness_m2.py"
click node_deployment_model "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/optimal_trading_model.zip"
click node_executor "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/trade_executor.py"
click node_dashboard "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/dashboard.py"
click node_credentials "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/.env.example"
click node_scheduler "https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/.github/workflows/daily_trading.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_data_pipeline,node_processed_data,node_rl_env,node_train,node_notebook,node_final_model,node_evaluations,node_evaluate,node_stress toneBlue
class node_deployment_model,node_executor,node_dashboard,node_credentials,node_scheduler,node_mock_execution toneAmber
class node_yahoo,node_alpaca toneMint
```

## Components

### Model Development

- [Feature pipeline](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/data_pipeline.py): Python pipeline · Builds engineered DJIA market dynamics. · `Optimized_RL_Trading_Strategy/data_pipeline.py`
- [Engineered market data](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/data/processed_market_dynamics.csv): CSV dataset · Feature contract for training and simulation. · `Optimized_RL_Trading_Strategy/data/processed_market_dynamics.csv`
- [Trading environment](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/custom_env.py): Gym environment · 567-dimensional simulation with risk and trading constraints. · `Optimized_RL_Trading_Strategy/custom_env.py`
- [PPO training](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/train_optimized.py): Training entry point · `Optimized_RL_Trading_Strategy/train_optimized.py`
- [Interactive workflow](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/main.ipynb): Jupyter notebook · `Optimized_RL_Trading_Strategy/main.ipynb`
- [Final PPO model](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/trained_models/final_model.zip): Model artifact · `Optimized_RL_Trading_Strategy/trained_models/final_model.zip`
- [Evaluation results](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/trained_models/evaluations.npz): Evaluation artifact · `Optimized_RL_Trading_Strategy/trained_models/evaluations.npz`
- [Policy evaluation](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/evaluate.py): Evaluation entry point · `Optimized_RL_Trading_Strategy/evaluate.py`
- [Stress validation](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/Optimized_RL_Trading_Strategy/stress_harness_m2.py): Stress harness · Validates policy resilience separately from standard evaluation. · `Optimized_RL_Trading_Strategy/stress_harness_m2.py`

### Paper Trading Deployment

- [Deployment model copy](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/optimal_trading_model.zip): Model artifact · Operational artifact that must remain aligned with training output. · `RL_Paper_Trading_Deployment/optimal_trading_model.zip`
- [Trade executor](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/trade_executor.py): Inference and execution · Computes target weights and submits or simulates paper trades. · `RL_Paper_Trading_Deployment/trade_executor.py`
- [Operator dashboard](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/dashboard.py): Streamlit app · Shows portfolio performance, daily P&L, and market regime. · `RL_Paper_Trading_Deployment/dashboard.py`
- [Credential setup](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/RL_Paper_Trading_Deployment/.env.example): Secrets documentation · Defines Alpaca credential configuration. · `RL_Paper_Trading_Deployment/.env.example`
- [Daily trading schedule](https://github.com/akisionov/reinforcement-learning-model-for-stock-market-data/blob/main/.github/workflows/daily_trading.yml): GitHub Actions workflow · `.github/workflows/daily_trading.yml`
- **Local mock execution**: Fallback executor · Used when Alpaca credentials are unavailable.

### External Services

- **Yahoo Finance**: Market data API · Supplies current market data through yfinance.
- **Alpaca paper trading**: Broker API

## Connections

- Feature pipeline → Engineered market data: writes
- Engineered market data → Trading environment: provides features
- Trading environment → PPO training: simulation interface
- Interactive workflow → PPO training: runs
- PPO training → Final PPO model: produces
- PPO training → Evaluation results: writes
- Final PPO model → Policy evaluation: evaluates
- Final PPO model → Stress validation: stress tests
- Final PPO model → Deployment model copy: version-aligned copy
- Deployment model copy → Trade executor: loads policy
- Yahoo Finance → Trade executor: current market data
- Credential setup → Trade executor: configures broker access
- Trade executor → Alpaca paper trading: paper orders
- Trade executor → Local mock execution: without credentials
- Operator dashboard → Trade executor: trading and portfolio view
- Daily trading schedule → Trade executor: runs daily