# hassentchoketch/optimal_monetary_policy_with_rl- architecture

> Architecture diagram of the GitHub repository hassentchoketch/optimal_monetary_policy_with_rl-, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hassentchoketch/optimal_monetary_policy_with_rl-
- Repository: https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-

## Overview

##### Project Type and Purpose
- This is a research-oriented RL pipeline, not a web or mobile app.
- Its goal is to ingest macroeconomic time series, estimate two economy models (SVAR & ANN), train DDPG agents on those models, run counterfactual analyses, and output tables/figures.

##### File Structure
Top-level modules:
- configs/        – hyperparameter and model settings (YAML)
- src/            – core code: data, environment, agents, policies, utils
- scripts/        – command-line entry points for each stage
- results/        – logs, tables, figures, checkpoints
- tests/          – unit tests
- main.py & setup.py – package entry and installation

##### Main Components
a. External Data Sources
- FRED, CBO, Wu-Xia API
b. Data Preparation Module
- scripts/data_preparation.py → src/data/data_loader.py
- Outputs cleaned/normalized series into data/ and results/checkpoints
c. Economy Estimation Module
- scripts/estimate_economy.py
- src/environment/* (base_economy, svar_economy, ann_economy)
- Outputs model parameters (pickles, PyTorch .pth)
d. RL Training Module
- scripts/train_agent.py (and train_all.sh)
- src/agents/* (ddpg_agent, networks, replay_buffer)
- src/policies/rl_policy.py & baseline_policies.py
- Consumes estimated economy models and configs → produces agent checkpoints & logs
e. Policy Evaluation Module
- scripts/evaluate_policy.py
- Runs historical/static simulations → writes tables, time series
f. Visualization & Reporting Module
- scripts/generate_figures.py
- src/utils/visualization.py & metrics.py → results/figures, results/tables
g. Configuration Manager
- configs/*.yaml loaded by each module via project_structur_template_creator.py or direct YAML reader
h. Logging & Utilities
- src/utils/logger.py for unified logging into results/logs
i. Tests
- tests/* verifying agents, environment, policies

##### Relationships & Interactions
- Data flows from external sources → Data Preparation → Saved intermediate files
- Estimation reads data → creates economy models → saves parameters
- Training reads economy parameters + configs → DDPG agent training → checkpoints & logs
- Evaluation uses checkpoints + economy models → generates simulation outputs
- Visualization reads outputs → produces figures & tables
- All stages read hyperparameters.yaml and models_config.yaml

##### Architectural Patterns & Design Principles
- Pipeline/Layers: each stage (prepare → estimate → train → evaluate → report) is a distinct layer
- Config-driven: centralize all hyperparameters and model settings in YAML
- Modular: each src sub-package (data, environment, agents, policies, utils) has single responsibility
- Reproducibility: fixed random seed, unit tests, logging

##### Relevant Technologies & Frameworks
- Python 3.10+, PyTorch 2.0+ for neural nets
- NumPy, Pandas for data handling
- YAML for configuration
- Matplotlib/Seaborn (via utils/visualization) for figures
- pytest for tests

##### Essential Elements to Capture
- Data sources & ingestion
- Config files feeding into every stage
- Two economy branches (SVAR & ANN) converging into training
- RL agent & replay buffer interaction
- Counterfactual evaluation branching into historical vs. static
- Output sinks: logs, tables, figures, checkpoints
- Utility services: logging, metrics, visualization

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Data Sources and Config Manager
    subgraph "External & Config" 
        direction TB
        DataSources["External Data Sources\n(FRED, CBO, Wu-Xia)"]:::external
        ConfigManager["Configuration Manager\n(YAML)"]:::external
    end

    %% Data Preparation Layer
    subgraph "Data Preparation" 
        direction TB
        DataPrepScript["scripts/data_preparation.py"]:::data
        DataLoader["Data Loader\n(src/data/data_loader.py)"]:::data
    end

    %% Estimation Layer
    subgraph "Economy Estimation" 
        direction TB
        EstScript["scripts/estimate_economy.py"]:::estimation
        BaseEconomy["Base Economy\n(src/environment/base_economy.py)"]:::estimation
        SVAREconomy["SVAR Economy\n(src/environment/svar_economy.py)"]:::estimation
        ANNEconomy["ANN Economy\n(src/environment/ann_economy.py)"]:::estimation
    end

    %% Training Layer
    subgraph "RL Training" 
        direction TB
        TrainScript["scripts/train_agent.py"]:::training
        DDPGAgent["DDPG Agent\n(src/agents/ddpg_agent.py)"]:::training
        Networks["Agent Networks\n(src/agents/networks.py)"]:::training
        ReplayBuffer["Replay Buffer\n(src/agents/replay_buffer.py)"]:::training
        RLPolicy["RL Policy\n(src/policies/rl_policy.py)"]:::training
        BaselinePolicy["Baseline Policies\n(src/policies/baseline_policies.py)"]:::training
    end

    %% Evaluation Layer
    subgraph "Policy Evaluation" 
        direction TB
        EvalScript["scripts/evaluate_policy.py"]:::evaluation
        StaticCF["Static Counterfactual\n(scripts/static_counterfactual.py)"]:::evaluation
    end

    %% Reporting Layer
    subgraph "Visualization & Reporting" 
        direction TB
        VizScript["scripts/generate_figures.py"]:::reporting
        VisualizationUtil["Visualization Utils\n(src/utils/visualization.py)"]:::utility
        MetricsUtil["Metrics Utils\n(src/utils/metrics.py)"]:::utility
        LoggerUtil["Logging Utility\n(src/utils/logger.py)"]:::utility
    end

    %% Results Sinks
    subgraph "Results Outputs"
        direction TB
        Logs["results/logs"]:::output
        Tables["results/tables"]:::output
        Figures["results/figures"]:::output
        Checkpoints["results/checkpoints"]:::output
    end

    %% Flow Connections
    DataSources -->|raw data| DataPrepScript
    ConfigManager --> DataPrepScript
    DataPrepScript -->|cleaned .csv| DataLoader
    DataLoader -->|normalized data| EstScript
    ConfigManager --> EstScript
    EstScript --> BaseEconomy
    BaseEconomy --> SVAREconomy
    BaseEconomy --> ANNEconomy
    SVAREconomy -->|model .pth/.pkl| TrainScript
    ANNEconomy -->|model .pth/.pkl| TrainScript
    ConfigManager --> TrainScript
    TrainScript --> DDPGAgent
    TrainScript --> Networks
    TrainScript --> ReplayBuffer
    TrainScript --> RLPolicy
    TrainScript --> BaselinePolicy
    DDPGAgent -->|agent .pth| Checkpoints
    Networks --> Checkpoints
    ReplayBuffer --> Checkpoints
    RLPolicy --> Checkpoints
    BaselinePolicy --> Checkpoints
    Checkpoints --> EvalScript
    SVAREconomy --> EvalScript
    ANNEconomy --> EvalScript
    ConfigManager --> EvalScript
    EvalScript --> StaticCF
    StaticCF --> Tables
    StaticCF --> Figures
    Tables --> VizScript
    Figures --> VizScript
    VizScript --> VisualizationUtil
    VizScript --> MetricsUtil
    VizScript --> LoggerUtil
    VisualizationUtil --> Figures
    MetricsUtil --> Tables
    LoggerUtil --> Logs

    %% Click Events
    click ConfigManager "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/configs/hyperparameters.yaml"
    click ConfigManager "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/configs/models_config.yaml"
    click ConfigManager "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/project_structur_template_creator.py"
    click DataLoader "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/data/data_loader.py"
    click EstScript "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/scripts/estimate_economy.py"
    click SVAREconomy "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/environment/svar_economy.py"
    click ANNEconomy "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/environment/ann_economy.py"
    click BaseEconomy "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/environment/base_economy.py"
    click TrainScript "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/scripts/train_agent.py"
    click DDPGAgent "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/agents/ddpg_agent.py"
    click Networks "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/agents/networks.py"
    click ReplayBuffer "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/agents/replay_buffer.py"
    click RLPolicy "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/policies/rl_policy.py"
    click BaselinePolicy "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/policies/baseline_policies.py"
    click EvalScript "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/scripts/evaluate_policy.py"
    click StaticCF "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/scripts/static_counterfactual.py"
    click VizScript "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/scripts/generate_figures.py"
    click VisualizationUtil "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/utils/visualization.py"
    click MetricsUtil "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/utils/metrics.py"
    click LoggerUtil "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/blob/main/src/utils/logger.py"
    click Logs "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/tree/main/results/logs"
    click Tables "https://github.com/hassentchoketch/optimal_monetary_policy_with_rl-/tree/main/results/tables"

    %% Style Definitions
    classDef external fill:#CFD8DC,stroke:#333,stroke-width:1px
    classDef data fill:#E3F2FD,stroke:#1E88E5,stroke-width:2px
    classDef estimation fill:#E8F5E9,stroke:#43A047,stroke-width:2px
    classDef training fill:#FFF3E0,stroke:#FB8C00,stroke-width:2px
    classDef evaluation fill:#F3E5F5,stroke:#8E24AA,stroke-width:2px
    classDef reporting fill:#FFEBEE,stroke:#E53935,stroke-width:2px
    classDef utility fill:#ECEFF1,stroke:#607D8B,stroke-width:1px,stroke-dasharray: 5 5
    classDef output fill:#FFFDE7,stroke:#FDD835,stroke-width:2px
```
