# ai4finance-foundation/finrl architecture

> Architecture diagram of the GitHub repository ai4finance-foundation/finrl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ai4finance-foundation/finrl
- Repository: https://github.com/ai4finance-foundation/finrl

## Overview

##### Project Type and Purpose
- FinRL is an open-source full-stack framework for automated trading based on financial reinforcement learning. It consists of an ecosystem where market environments (data sources, processors, simulated trading environments) feed into intelligent agents using various RL algorithms. These agents, in turn, drive applications that translate decisions into trading strategies.
- The README clearly outlines the “three-layer” concept (market environments, agents, applications) and emphasizes a train-test-trade pipeline. The file tree reinforces this with key directories (finrl/agents, finrl/meta, finrl/applications) and main pipeline scripts (train.py, test.py, trade.py).

##### File Structure to Identify Components
- Top-Level Directories/Files:
- /finrl: Contains the main implementation.
- applications: Contains real-world trading examples (e.g., Stock_NeurIPS2018, cryptocurrency_trading). This represents the “application” or front-facing layer.
- agents: Contains various reinforcement learning algorithms, segregated by framework (e.g., elegantrl, stablebaselines3, rllib). This is the core “agent” layer.
- meta: Consists of market simulation environments, data processors (for different data sources like Yahoo, Alpaca, etc.) and configuration files that set up the data-handling logic. This represents the “market environment” layer.
- Pipeline Scripts (train.py, test.py, trade.py): Orchestrate the workflow from data ingestion/training to simulation and live (paper) trading.
- Other folders such as docker (containerization scripts), docs (documentation), examples, and unit_tests help with deployment, demos, and code testing.

- Market/Data Environment:
- Components: finrl/meta (data processors, market simulation environments, preprocessor, environment definitions for stocks, crypto, portfolio allocation, etc.).
- Interactions: Gathers raw data from multiple external data sources (as listed in README) and pre-processes it into formats suitable for modeling.
- Intelligent Agents:
- Components: finrl/agents (with submodules for elegantRL, rllib, stablebaselines3).
- Role: Implements reinforcement learning algorithms that make sequential decisions based on processed market data.
- Technologies: Built using frameworks like PyTorch, Stable Baselines3, etc.
- Trading Applications:
- Components: finrl/applications (various trading strategy implementations, e.g., Stock_NeurIPS2018, stock_trading, imitation_learning).
- Role: Utilize agent outputs to simulate or execute trading decisions using train-test-trade scripts.
- Pipeline Orchestration:
- Components: train.py, test.py, trade.py which tie together the flow from market environments to agent decision-making and then to applications.
- Infrastructure/Deployment:
- Components: Docker (with container build and deployment scripts) and CI/CD workflows under the .github folder.
- Ensures reproducible environments and automated testing.
- Documentation & Examples:
- Components: docs and examples directories.
- Provide the usage guides and tutorials to understand and extend the framework.

b. Relationships and Interactions:
- Data flows upward from the Market Environment (finrl/meta) to Agents (finrl/agents) which then drive Applications (finrl/applications).
- The training pipeline (via train.py) pulls in data processed by the meta layer, feeds it to the agents for model training, and then the trained agents are utilized by test.py and trade.py to simulate or execute trading strategies.
- The docker container and CI/CD workflows encapsulate these interactions for deployment and automated testing.

##### Architectural Patterns and Design Principles
- Layered Architecture: Clearly segmented into three layers (data/environment, agent, application). This separation of concerns makes the system modular and extensible.
- Pipeline-based Design: A sequential flow (train-test-trade) drives the data and control flow through the system.
- Modularity and Extensibility: The separation of agents into different submodules (e.g., elegantrl, rllib, stablebaselines3) and the extensive use of external data processors show a design open to extension.
- Integration with External Systems: Support for various data sources and containerization with Docker indicate that the system is built with integration and deployment in mind.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data Source
    ED["External Data Sources"]:::external

    %% Main Layers
    ME["Market Environment\n(finrl/meta)"]:::market
    IA["Intelligent Agents\n(finrl/agents)"]:::agent
    TA["Trading Applications\n(finrl/applications)"]:::application

    %% Pipeline Orchestration Subgraph
    subgraph "Pipeline Orchestration"
        T1["train.py"]:::pipeline
        T2["test.py"]:::pipeline
        T3["trade.py"]:::pipeline
    end

    %% Infrastructure & Deployment Subgraph
    subgraph "Infrastructure & Deployment"
        D1["Docker"]:::infra
        D2["GitHub Workflows\n(.github)"]:::infra
    end

    %% Documentation & Examples Subgraph
    subgraph "Documentation & Examples"
        DOC["Docs"]:::docs
        EX["Examples"]:::docs
    end

    %% Relationships
    ED -->|"data"| ME
    ME -->|"processedData"| IA
    IA -->|"agentDecisions"| TA

    %% Pipeline Connections
    T1 -->|"loadsData"| ME
    T1 -->|"trains"| IA
    T2 -->|"testsAgents"| IA
    T2 -->|"simulatesTrading"| TA
    T3 -->|"executesTrades"| TA

    %% Infrastructure supports Pipeline and Layers
    D1 ---|"containerized"| T1
    D1 --- T2
    D1 --- T3
    D1 --- ME
    D1 --- IA
    D1 --- TA
    D2 ---|"CI/CD"| T1
    D2 --- T2
    D2 --- T3

    %% Class Definitions for styling
    classDef market fill:#9ecae1,stroke:#3182bd,color:#ffffff,stroke-width:2px;
    classDef agent fill:#a1d99b,stroke:#31a354,color:#ffffff,stroke-width:2px;
    classDef application fill:#fdae6b,stroke:#e6550d,color:#ffffff,stroke-width:2px;
    classDef pipeline fill:#d9d9d9,stroke:#636363,color:#000000,stroke-width:2px;
    classDef infra fill:#c7e9c0,stroke:#31a354,color:#000000,stroke-width:2px,stroke-dasharray: 5,5;
    classDef docs fill:#fdd0a2,stroke:#fdae6b,color:#000000,stroke-width:2px;
    classDef external fill:#d0d0d0,stroke:#808080,color:#000000,stroke-width:2px;

    %% Click Events
    click ME "https://github.com/ai4finance-foundation/finrl/tree/master/finrl/meta"
    click IA "https://github.com/ai4finance-foundation/finrl/tree/master/finrl/agents"
    click TA "https://github.com/ai4finance-foundation/finrl/tree/master/finrl/applications"
    click T1 "https://github.com/ai4finance-foundation/finrl/blob/master/finrl/train.py"
    click T2 "https://github.com/ai4finance-foundation/finrl/blob/master/finrl/test.py"
    click T3 "https://github.com/ai4finance-foundation/finrl/blob/master/finrl/trade.py"
    click D1 "https://github.com/ai4finance-foundation/finrl/tree/master/docker"
    click D2 "https://github.com/ai4finance-foundation/finrl/blob/master/.github"
    click DOC "https://github.com/ai4finance-foundation/finrl/tree/master/docs"
    click EX "https://github.com/ai4finance-foundation/finrl/tree/master/examples"
```
