How ziadfrancis/reinforcement_trading_part_2 works
1. This is a file-based research pipeline for XAUUSD minute data, not an online service. The core flow is: raw MT4/MT5 CSV → validated time-aligned bars → causal features → trading environment / baselines → walk-forward evaluation → saved artifacts and charts.
2. config.py is the central boundary for experiment setup. It controls the source dataset, timezones, resampling/decision timeframe, split strategy, walk-forward fold counts, PPO settings, and the default paths used by the rest of the pipeline.
3. data_loader.py owns the ingestion layer. It parses broker-style CSVs, handles EET/EEST time normalization, shifts M1 candle timestamps to candle-close semantics, validates continuity/shape, and prepares data for resampling into higher decision bars while retaining M1 for intrabar execution simulation.
4. features.py is the feature-engineering subsystem. It builds the causal, mostly stationary observation matrix used by both baselines and RL, with ATR/price-normalized distances, ratios, session flags, and candle-shape features. The README indicates the final observation set is 25 features, with some highly collinear candidates intentionally removed.
5. env_bracket.py defines the main simulation boundary: a Gymnasium-compatible bracket-trading environment. It simulates entry/exit logic with direction, stop-loss, and take-profit buckets; uses M1 data to resolve intrabar TP/SL hits; assumes SL-first when both are touched in the same candle; and uses fixed-fractional risk sizing with reward normalized by risk rather than raw PnL.
6. baselines.py provides non-RL reference policies, likely including random and rule-based trend-following/EMA-ATR strategies. These are used to sanity-check the pipeline and compare RL performance against simple deterministic or stochastic behaviors.
7. train_ppo.py is the RL training subsystem. It trains PPO over the bracket-action environment with a MultiDiscrete([direction, SL bucket, TP/R bucket]) action space. The README makes clear there are two validation modes: a sliding walk-forward mode that repeatedly retrains and tests on successive out-of-sample windows, and a block walk-forward mode that seals a final holdout and rolls train/validation folds over the rest.
8. run_pipeline.py is the main pre-test orchestration entry point. It performs the temporal train/validation/test split, runs the non-RL pipeline end to end, and writes validation-only outputs so the test split stays sealed.
9. final_holdout_eval.py is the explicit one-time release gate for the test set. It is the architecture’s “single reveal” boundary: after model selection is frozen, this script evaluates on the sealed holdout and produces the final report.
10. evaluate.py and model_artifacts.py are the reporting and persistence layer. evaluate.py computes trade-level and equity-level metrics such as drawdown and summary statistics, while model_artifacts.py likely standardizes how trained models, fold outputs, and CSV reports are stored.
11. visualize.py and the notebooks are the analysis layer. They generate charts for candles, indicators, volatility, sessions, feature correlations, trades, equity curves, and drawdowns, while the notebooks provide exploratory comparison across consistency checks, ensembles, and test/fold analyses.
12. leakage_checks.py and training_diagnostics.py enforce the research boundary against lookahead bias and overfitting. The repo emphasizes future-append stability checks, sealed test splits by default, and fold-level diagnostics so that validation behavior is measured before any final holdout exposure.
13. Overall, the architecture is organized around strict temporal boundaries: raw market history is transformed into causal features, the environment simulates execution with pessimistic intrabar assumptions, and evaluation is done through walk-forward or sealed-holdout protocols. That makes the repo suitable for a diagram with clear layers for ingestion, feature generation, simulation, training, evaluation, and artifact/report output.
Main entry points: evaluate.py Evaluate · config.py Config · features.py Features · train_ppo.py Train PPO
Components 15
Setup
Central experiment configuration and defaults.
- Config · Defines dataset, time handling, splits, walk-forward settings, and default paths.
config.py
Ingestion
Raw CSV parsing, temporal normalization, and feature construction.
- Data Loader · Parses broker CSVs, normalizes timestamps, validates continuity, and prepares M1/bars.
data_loader.py - Features · Builds stationary observation features for baselines and RL.
features.py
Simulation
Trading environment and baseline policies.
- Bracket Env · Simulates bracket trading with intrabar TP/SL resolution and risk-normalized rewards.
env_bracket.py - Baselines · Non-RL benchmark strategies for sanity checks and comparison.
baselines.py
Training
PPO training and orchestration.
- Train PPO · Trains PPO on the bracket environment with discrete direction and risk buckets.
train_ppo.py - Run Pipeline · Runs the pre-test workflow, including split setup and validation-only execution.
run_pipeline.py
Evaluation
Walk-forward testing, holdout scoring, and persisted outputs.
- Holdout Eval · Performs the one-time final evaluation on the sealed holdout set.
final_holdout_eval.py - Evaluate · Computes trade-level and equity-level performance statistics.
evaluate.py - Artifacts · Standardizes saved models, fold outputs, and CSV reports.
model_artifacts.py
Quality
Leakage prevention and diagnostics.
- Leak Checks · Detects lookahead and future-append leakage across temporal splits.
leakage_checks.py - Diagnostics · Tracks fold-level behavior and overfitting signals during training.
training_diagnostics.py
Analysis
Charts, result inspection, and notebook exploration.
- Visualize · Generates charts for candles, indicators, equity, drawdowns, and trade outcomes.
visualize.py - Results View · Inspects stored experiment outputs and summaries.
view_results.py - Notebooks · Interactive analyses for consistency checks, ensemble comparison, and fold/test review.
Connections 26
- Config → Data Loader: dataset/timebase · Provides source, timezone, and resampling settings.
- Config → Features: feature setup · Controls observation construction and experiment defaults.
- Config → Bracket Env: risk/env params · Sets trading and decision-timeframe assumptions.
- Config → Train PPO: ppo params · Defines training hyperparameters and fold strategy.
- Config → Run Pipeline: split policy · Drives train/validation/test orchestration.
- Data Loader → Features: bars in · Feeds validated and time-aligned market data into feature generation.
- Data Loader → Bracket Env: m1 feed · Supplies minute data for intrabar TP/SL execution simulation.
- Features → Bracket Env: observations · Provides the causal state used by the agent and baselines.
- Features → Baselines: inputs · Supplies indicators and state variables for rule-based policies.
- Bracket Env → Train PPO: rl loop · Defines the action/reward interaction for PPO.
- Baselines → Evaluate: benchmarks · Produces reference trade streams for comparison.
- Train PPO → Evaluate: test runs · Sends trained policies into evaluation.
- Run Pipeline → Baselines: non-rl run · Executes baseline pipeline end to end.
- Run Pipeline → Evaluate: validation metrics · Computes validation-only performance summaries.
- Evaluate → Artifacts: reports · Persists metrics and fold outputs.
- Train PPO → Artifacts: checkpoints · Stores learned policies and training artifacts.
- Holdout Eval → Evaluate: holdout scoring · Runs final metrics on the sealed test set.
- Holdout Eval → Artifacts: final outputs · Writes the final frozen reports and artifacts.
- Evaluate → Visualize: metrics plots · Feeds equity and trade statistics into charts.
- Artifacts → Results View: saved outputs · Provides stored runs for inspection.
- Artifacts → Notebooks: analysis data · Supports exploratory comparison and post-run analysis.
- Leak Checks → Data Loader: split sanity · Checks temporal integrity around ingestion and alignment.
- Leak Checks → Features: causality · Verifies features do not leak future information.
- Diagnostics → Train PPO: fit signals · Monitors training stability and overfitting.
- Diagnostics → Evaluate: fold review · Summarizes validation behavior across walk-forward folds.
- Visualize → Notebooks: shared analysis · Charts complement notebook-based exploration.