How hassentchoketch/optimal_monetary_policy_with_rl- works
Project Type and Purpose
• This is a research-oriented RL pipeline, not a web or mobile app.
• Its goal is to ingest macroeconomic time series, estimate two economy models (SVAR & ANN), train DDPG agents on those models, run counterfactual analyses, and output tables/figures.
File Structure
Top-level modules:
• configs/ – hyperparameter and model settings (YAML)
• src/ – core code: data, environment, agents, policies, utils
• scripts/ – command-line entry points for each stage
• results/ – logs, tables, figures, checkpoints
• tests/ – unit tests
• main.py & setup.py – package entry and installation
Main Components
a. External Data Sources
• FRED, CBO, Wu-Xia API
b. Data Preparation Module
• scripts/data_preparation.py → src/data/data_loader.py
• Outputs cleaned/normalized series into data/ and results/checkpoints
c. Economy Estimation Module
• scripts/estimate_economy.py
• src/environment/* (base_economy, svar_economy, ann_economy)
• Outputs model parameters (pickles, PyTorch .pth)
d. RL Training Module
• scripts/train_agent.py (and train_all.sh)
• src/agents/* (ddpg_agent, networks, replay_buffer)
• src/policies/rl_policy.py & baseline_policies.py
• Consumes estimated economy models and configs → produces agent checkpoints & logs
e. Policy Evaluation Module
• scripts/evaluate_policy.py
• Runs historical/static simulations → writes tables, time series
f. Visualization & Reporting Module
• scripts/generate_figures.py
• src/utils/visualization.py & metrics.py → results/figures, results/tables
g. Configuration Manager
• configs/*.yaml loaded by each module via project_structur_template_creator.py or direct YAML reader
h. Logging & Utilities
• src/utils/logger.py for unified logging into results/logs
i. Tests
• tests/* verifying agents, environment, policies
Relationships & Interactions
• Data flows from external sources → Data Preparation → Saved intermediate files
• Estimation reads data → creates economy models → saves parameters
• Training reads economy parameters + configs → DDPG agent training → checkpoints & logs
• Evaluation uses checkpoints + economy models → generates simulation outputs
• Visualization reads outputs → produces figures & tables
• All stages read hyperparameters.yaml and models_config.yaml
Architectural Patterns & Design Principles
• Pipeline/Layers: each stage (prepare → estimate → train → evaluate → report) is a distinct layer
• Config-driven: centralize all hyperparameters and model settings in YAML
• Modular: each src sub-package (data, environment, agents, policies, utils) has single responsibility
• Reproducibility: fixed random seed, unit tests, logging
Relevant Technologies & Frameworks
• Python 3.10+, PyTorch 2.0+ for neural nets
• NumPy, Pandas for data handling
• YAML for configuration
• Matplotlib/Seaborn (via utils/visualization) for figures
• pytest for tests
Essential Elements to Capture
• Data sources & ingestion
• Config files feeding into every stage
• Two economy branches (SVAR & ANN) converging into training
• RL agent & replay buffer interaction
• Counterfactual evaluation branching into historical vs. static
• Output sinks: logs, tables, figures, checkpoints
• Utility services: logging, metrics, visualization