# ai4finance-foundation/finrl_crypto architecture

> Architecture diagram of the GitHub repository ai4finance-foundation/finrl_crypto, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ai4finance-foundation/finrl_crypto
- Repository: https://github.com/ai4finance-foundation/finrl_crypto

## Overview

1. **Repository purpose**
   - This is a research/trading pipeline for cryptocurrency reinforcement learning, centered on Binance data and DRL agents.
   - The codebase is organized as a sequence of offline experiment scripts rather than a long-running service: collect data, train/optimize, validate, backtest, and score robustness.

2. **Top-level execution flow**
   - The numbered scripts represent the main experiment stages:
     - `0_dl_trainval_data.py` / `0_dl_trade_data.py` for data acquisition
     - `1_optimize_cpcv.py`, `1_optimize_kcv.py`, `1_optimize_wf.py` for hyperparameter search under different validation schemes
     - `2_validate.py` for out-of-sample validation
     - `4_backtest.py` for strategy backtesting
     - `5_pbo.py` for probability-of-backtest-overfitting analysis
   - This suggests a pipeline designed to reduce overfitting before deploying an agent.

3. **Configuration boundary**
   - `config_main.py` holds experiment parameters such as tickers, timeframe, date ranges, technical indicators, and likely training/backtest settings.
   - `config_api.py` isolates exchange credentials, which is a clear boundary between runtime logic and secrets.
   - This separation matters because most modules appear to read shared configuration rather than own their own hardcoded settings.

4. **Data ingestion and feature engineering**
   - `processor_Base.py` defines shared preprocessing behavior.
   - `processor_Binance.py` is the main market data adapter for Binance.
   - `processor_Yahoo.py` provides an alternate market data source, likely for comparison or fallback.
   - These processors form the data boundary: raw exchange/API data enters here and is converted into model-ready tabular features.

5. **Trading environments**
   - `environment_CCXT.py` and `environment_Alpaca.py` provide exchange-facing environments.
   - `environment_CCXT.py` is the most relevant for this repo’s crypto focus and likely serves as a bridge to many exchanges through CCXT.
   - `environment_Alpaca.py` is present as a separate boundary, indicating the project can be adapted beyond Binance even though the README emphasizes Binance.

6. **Reinforcement learning agent layer**
   - `drl_agents/agents/` contains algorithm-specific implementations for A2C, PPO, DDPG, TD3, and SAC.
   - `AgentBase.py` and `net.py` suggest a common agent abstraction and shared neural network components.
   - `elegantrl_models.py` implies the project is built around or adapted from ElegantRL-style training patterns.
   - This layer is where action selection, policy/value learning, and algorithm-specific updates are separated from market data and evaluation.

7. **Training runtime**
   - The `train/` package is the execution engine for model learning.
   - `train/run.py` likely coordinates experiments, while `worker.py`, `learner.py`, `evaluator.py`, and `replay_buffer.py` split the standard RL responsibilities into collection, optimization, evaluation, and storage.
   - `train/config.py`, `train/utils.py`, and `train/sandbox.py` provide support for experiment setup and isolated runs.
   - This is the primary internal subsystem for running DRL training loops.

8. **Validation and model selection logic**
   - `function_train_test.py`, `function_CPCV.py`, and `function_PBO.py` implement the statistical validation scaffolding around the agents.
   - The repo explicitly supports:
     - Combinatorial Purged Cross-Validation
     - K-fold cross-validation
     - Walk-forward validation
     - PBO scoring
   - These functions are a distinct evaluation boundary: they assess robustness of strategies rather than just training reward.

9. **Performance and financial metrics**
   - `function_finance_metrics.py` computes trading-oriented evaluation measures such as return and risk statistics.
   - The existence of a separate metrics module indicates backtest outputs are not treated as generic ML metrics; they are finance-specific and likely feed into the validation/backtest scripts.

10. **Backtesting and experiment outputs**
   - `4_backtest.py` is the strategy simulation stage after validation.
   - The README mentions plots and metrics saved under `/plots_and_metrics/`, so the project produces offline artifacts rather than serving live predictions.
   - This makes the output boundary clear: trained policy -> simulated trading -> metrics/plots for comparison.

11. **External dependencies and integrations**
   - Binance is the primary external exchange integration.
   - CCXT support is important because it broadens the exchange interface abstraction beyond a single venue.
   - TA-Lib and pandas/numpy compatibility are material because they affect feature calculation and preprocessing reliability.

12. **How the pieces fit together**
   - Market data is pulled through exchange-specific processors, converted into engineered features, then fed to DRL environments and agents.
   - Training happens inside the `train/` package, while robustness is enforced through specialized validation utilities and multiple cross-validation schemes.
   - Final decisions are based on backtests and financial metrics, with PBO used to estimate whether the apparent performance is likely overfit.

Main entry points: [`drl_agents/agents/AgentBase.py`](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentBase.py) (Agent base), [`processor_Binance.py`](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/processor_Binance.py) (Binance processor), [`function_train_test.py`](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/function_train_test.py) (Validation funcs), [`train/run.py`](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/train/run.py) (Train run)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["Experiment pipeline"]
  node_backtest_script["Backtest<br/>simulation stage<br/>[4_backtest.py]"]
  node_pbo_script["PBO<br/>overfitting score<br/>[5_pbo.py]"]
  node_data_collect_trainval["Train/val data<br/>data collection script"]
  node_data_collect_trade["Trade data<br/>data collection script<br/>[0_dl_trade_data.py]"]
  node_opt_cpcv["Opt CPCV<br/>hyperparam search<br/>[1_optimize_cpcv.py]"]
  node_opt_kcv["Opt KCV<br/>hyperparam search<br/>[1_optimize_kcv.py]"]
  node_opt_wf["Opt WF<br/>hyperparam search<br/>[1_optimize_wf.py]"]
  node_validate_script["Validate<br/>oos validation<br/>[2_validate.py]"]
end

subgraph group_data["Data boundary"]
  node_config_main["Main config<br/>experiment config<br/>[config_main.py]"]
  node_config_api["API config<br/>secrets config<br/>[config_api.py]"]
  node_processor_base["Base processor<br/>preprocessing base<br/>[processor_Base.py]"]
  node_processor_binance["Binance processor<br/>market adapter"]
  node_processor_yahoo["Yahoo processor<br/>market adapter<br/>[processor_Yahoo.py]"]
  node_env_ccxt["CCXT env<br/>trading environment"]
  node_env_alpaca["Alpaca env<br/>trading environment"]
end

subgraph group_rl["RL core"]
  node_agent_base["Agent base<br/>agent abstraction<br/>[AgentBase.py]"]
  node_agent_nets["Policy nets<br/>neural modules<br/>[net.py]"]
  node_agent_a2c["A2C<br/>policy-gradient agent<br/>[AgentA2C.py]"]
  node_agent_ppo["PPO<br/>policy-gradient agent<br/>[AgentPPO.py]"]
  node_agent_ddpg["DDPG<br/>actor-critic agent<br/>[AgentDDPG.py]"]
  node_agent_td3["TD3<br/>actor-critic agent<br/>[AgentTD3.py]"]
  node_agent_sac["SAC<br/>actor-critic agent<br/>[AgentSAC.py]"]
  node_train_run["Train run<br/>training orchestrator<br/>[run.py]"]
  node_train_core["Train core<br/>rl runtime"]
end

subgraph group_eval["Evaluation"]
  node_validation_funcs["Validation funcs<br/>robustness evaluation"]
  node_finance_metrics["Finance metrics<br/>performance metrics"]
end

node_config_main -->|"parameters"| node_data_collect_trainval
node_config_main -->|"parameters"| node_data_collect_trade
node_config_api -->|"credentials"| node_processor_binance
node_processor_base -->|"shared logic"| node_processor_binance
node_processor_base -->|"shared logic"| node_processor_yahoo
node_processor_binance -->|"features"| node_env_ccxt
node_processor_yahoo -->|"features"| node_env_alpaca
node_env_ccxt -->|"experience"| node_agent_base
node_env_alpaca -->|"experience"| node_agent_base
node_agent_base -->|"uses"| node_agent_nets
node_agent_base -->|"specializes"| node_agent_a2c
node_agent_base -->|"specializes"| node_agent_ppo
node_agent_base -->|"specializes"| node_agent_ddpg
node_agent_base -->|"specializes"| node_agent_td3
node_agent_base -->|"specializes"| node_agent_sac
node_train_run -->|"orchestrates"| node_train_core
node_train_core -->|"trains"| node_agent_base
node_train_core -->|"interacts"| node_env_ccxt
node_validation_funcs -->|"supports"| node_opt_cpcv
node_validation_funcs -->|"supports"| node_opt_kcv
node_validation_funcs -->|"supports"| node_opt_wf
node_validation_funcs -->|"drives"| node_validate_script
node_finance_metrics -->|"scores"| node_validate_script
node_finance_metrics -->|"scores"| node_backtest_script
node_data_collect_trainval -->|"ingests"| node_processor_binance
node_data_collect_trade -->|"ingests"| node_processor_binance
node_opt_cpcv -->|"configures"| node_train_run
node_opt_kcv -->|"configures"| node_train_run
node_opt_wf -->|"configures"| node_train_run
node_validate_script -->|"feeds"| node_backtest_script
node_backtest_script -->|"assesses"| node_pbo_script

click node_config_main "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/config_main.py"
click node_config_api "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/config_api.py"
click node_processor_base "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/processor_Base.py"
click node_processor_binance "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/processor_Binance.py"
click node_processor_yahoo "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/processor_Yahoo.py"
click node_env_ccxt "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/environment_CCXT.py"
click node_env_alpaca "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/environment_Alpaca.py"
click node_agent_base "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentBase.py"
click node_agent_nets "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/net.py"
click node_agent_a2c "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentA2C.py"
click node_agent_ppo "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentPPO.py"
click node_agent_ddpg "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentDDPG.py"
click node_agent_td3 "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentTD3.py"
click node_agent_sac "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentSAC.py"
click node_train_run "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/train/run.py"
click node_validation_funcs "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/function_train_test.py"
click node_finance_metrics "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/function_finance_metrics.py"
click node_backtest_script "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/4_backtest.py"
click node_pbo_script "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/5_pbo.py"
click node_data_collect_trainval "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/0_dl_trainval_data.py"
click node_data_collect_trade "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/0_dl_trade_data.py"
click node_opt_cpcv "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/1_optimize_cpcv.py"
click node_opt_kcv "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/1_optimize_kcv.py"
click node_opt_wf "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/1_optimize_wf.py"
click node_validate_script "https://github.com/ai4finance-foundation/finrl_crypto/blob/master/2_validate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_backtest_script,node_pbo_script,node_data_collect_trainval,node_data_collect_trade,node_opt_cpcv,node_opt_kcv,node_opt_wf,node_validate_script toneBlue
class node_config_main,node_config_api,node_processor_base,node_processor_binance,node_processor_yahoo,node_env_ccxt,node_env_alpaca toneAmber
class node_agent_base,node_agent_nets,node_agent_a2c,node_agent_ppo,node_agent_ddpg,node_agent_td3,node_agent_sac,node_train_run,node_train_core toneMint
class node_validation_funcs,node_finance_metrics toneRose
```

## Components

### Experiment pipeline

Offline scripts that run the research workflow end to end.

- [Backtest](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/4_backtest.py): simulation stage · Offline strategy simulation after validation. · `4_backtest.py`
- [PBO](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/5_pbo.py): overfitting score · Probability-of-backtest-overfitting analysis stage. · `5_pbo.py`
- [Train/val data](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/0_dl_trainval_data.py): data collection script · Acquires data for training and validation. · `0_dl_trainval_data.py`
- [Trade data](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/0_dl_trade_data.py): data collection script · Acquires data for trading-oriented runs. · `0_dl_trade_data.py`
- [Opt CPCV](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/1_optimize_cpcv.py): hyperparam search · Optimization under combinatorial purged cross-validation. · `1_optimize_cpcv.py`
- [Opt KCV](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/1_optimize_kcv.py): hyperparam search · Optimization under k-fold cross-validation. · `1_optimize_kcv.py`
- [Opt WF](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/1_optimize_wf.py): hyperparam search · Optimization under walk-forward validation. · `1_optimize_wf.py`
- [Validate](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/2_validate.py): oos validation · Out-of-sample validation stage. · `2_validate.py`

### Data boundary

Configuration, ingestion, feature engineering, and trading environments.

- [Main config](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/config_main.py): experiment config · Shared experiment parameters like tickers, dates, indicators, and training settings. · `config_main.py`
- [API config](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/config_api.py): secrets config · Exchange credentials and other sensitive integration settings. · `config_api.py`
- [Base processor](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/processor_Base.py): preprocessing base · Shared preprocessing and data-shaping behavior. · `processor_Base.py`
- [Binance processor](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/processor_Binance.py): market adapter · Primary Binance data ingestion and feature preparation path. · `processor_Binance.py`
- [Yahoo processor](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/processor_Yahoo.py): market adapter · Alternate market data source used for comparison or fallback. · `processor_Yahoo.py`
- [CCXT env](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/environment_CCXT.py): trading environment · Exchange-facing environment abstraction for crypto and other CCXT-supported venues. · `environment_CCXT.py`
- [Alpaca env](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/environment_Alpaca.py): trading environment · Separate broker/exchange environment boundary for Alpaca integration. · `environment_Alpaca.py`

### RL core

Agents and training runtime for policy learning.

- [Agent base](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentBase.py): agent abstraction · Common interface and shared RL agent behavior. · `drl_agents/agents/AgentBase.py`
- [Policy nets](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/net.py): neural modules · Shared network components used by multiple agents. · `drl_agents/agents/net.py`
- [A2C](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentA2C.py): policy-gradient agent · Advantage actor-critic implementation. · `drl_agents/agents/AgentA2C.py`
- [PPO](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentPPO.py): policy-gradient agent · Proximal policy optimization implementation. · `drl_agents/agents/AgentPPO.py`
- [DDPG](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentDDPG.py): actor-critic agent · Deterministic policy gradient implementation. · `drl_agents/agents/AgentDDPG.py`
- [TD3](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentTD3.py): actor-critic agent · Twin delayed deep deterministic policy gradient implementation. · `drl_agents/agents/AgentTD3.py`
- [SAC](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/drl_agents/agents/AgentSAC.py): actor-critic agent · Soft actor-critic implementation. · `drl_agents/agents/AgentSAC.py`
- [Train run](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/train/run.py): training orchestrator · Coordinates the main learning loop and experiment execution. · `train/run.py`
- **Train core**: rl runtime · Worker, learner, evaluator, replay buffer, and training utilities.

### Evaluation

Validation, backtesting, and finance-oriented scoring.

- [Validation funcs](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/function_train_test.py): robustness evaluation · Train/test, CPCV, and PBO scaffolding for robustness analysis. · `function_train_test.py`
- [Finance metrics](https://github.com/ai4finance-foundation/finrl_crypto/blob/master/function_finance_metrics.py): performance metrics · Trading-specific return and risk calculations. · `function_finance_metrics.py`

## Connections

- Main config → Train/val data: parameters · Provides tickers, dates, indicators, and experiment settings.
- Main config → Trade data: parameters · Provides shared run configuration.
- API config → Binance processor: credentials · Supplies exchange access settings.
- Base processor → Binance processor: shared logic · Binance processor extends common preprocessing behavior.
- Base processor → Yahoo processor: shared logic · Yahoo processor extends common preprocessing behavior.
- Binance processor → CCXT env: features · Feeds model-ready market data into the trading environment.
- Yahoo processor → Alpaca env: features · Feeds alternate market data into the broker environment.
- CCXT env → Agent base: experience · Environment observations and rewards drive agent interaction.
- Alpaca env → Agent base: experience · Environment observations and rewards drive agent interaction.
- Agent base → Policy nets: uses · Agents rely on shared neural network components.
- Agent base → A2C: specializes · A2C implementation under the common agent interface.
- Agent base → PPO: specializes · PPO implementation under the common agent interface.
- Agent base → DDPG: specializes · DDPG implementation under the common agent interface.
- Agent base → TD3: specializes · TD3 implementation under the common agent interface.
- Agent base → SAC: specializes · SAC implementation under the common agent interface.
- Train run → Train core: orchestrates · Run script coordinates the training runtime.
- Train core → Agent base: trains · Learner and worker infrastructure optimize agents.
- Train core → CCXT env: interacts · Collects experience from the trading environment.
- Validation funcs → Opt CPCV: supports · Implements combinatorial purged cross-validation logic.
- Validation funcs → Opt KCV: supports · Implements k-fold validation logic.
- Validation funcs → Opt WF: supports · Implements walk-forward validation logic.
- Validation funcs → Validate: drives · Provides robustness evaluation scaffolding.
- Finance metrics → Validate: scores · Computes trading performance metrics for validation results.
- Finance metrics → Backtest: scores · Computes return and risk statistics for backtest outputs.
- Train/val data → Binance processor: ingests · Builds training and validation datasets from Binance data.
- Trade data → Binance processor: ingests · Builds trade-oriented datasets from Binance data.
- Opt CPCV → Train run: configures · Runs training under CPCV search.
- Opt KCV → Train run: configures · Runs training under k-fold search.
- Opt WF → Train run: configures · Runs training under walk-forward search.
- Validate → Backtest: feeds · Validated strategy parameters flow into backtesting.
- Backtest → PBO: assesses · Backtest results are analyzed for overfitting risk.