How areshglad/monte-carlo-portfolio-risk-engine works
1. This repository is a layered portfolio-risk analytics pipeline, centered on dbt SQL models plus a Python Monte Carlo engine. The overall purpose is to turn market price data into risk metrics, simulation outputs, and portfolio optimization results.
2. The architecture is organized in a classic Bronze / Silver / Gold pattern:
- Bronze: raw market price ingestion and normalization.
- Silver: intermediate market-statistics transformations.
- Gold: analytical outputs used for risk reporting and optimization.
3. The Bronze layer is represented by bronze_market_prices.sql. This is the raw-data boundary of the system, where market price data is staged before any portfolio analytics are computed. source.yml likely defines the upstream source tables used by dbt.
4. The Silver layer consists of:
- silver_daily_returns.sql
- silver_rolling_volatility.sql
- silver_correlation_base.sql
These models derive core statistical inputs from prices: returns, volatility over time windows, and the correlation structure needed for covariance and portfolio risk calculations.
5. The Gold layer contains the business/analytics outputs:
- gold_covariance_input.sql
- gold_monte_carlo_inputs.sql
- gold_portfolio_risk_metrics.sql
- gold_var_summary.sql
- gold_simulation_summary.sql
- gold_efficient_frontier.sql
This is where the model becomes decision-oriented: the system prepares simulation inputs, computes portfolio risk measures, summarizes Monte Carlo results, and derives an efficient frontier.
6. monte_carlo_engine.py is the main procedural/runtime component outside dbt. It likely consumes the curated Gold inputs and runs the simulation logic that dbt SQL alone is not suited for, such as iterative random-path generation, portfolio outcome sampling, and VaR/CVaR-style calculations.
7. The data flow appears to be one-way and sequential:
raw market prices → daily returns / volatility / correlation → covariance and simulation inputs → Monte Carlo simulation and portfolio risk summaries → frontier/risk outputs for analysis and reporting.
8. The repository includes generated or downstream artifacts:
- monte_carlo_simulation_results.csv
- monte carlo visuals.pdf
These indicate that the Python layer produces tabular simulation results and a visual/reporting deliverable, separate from the dbt transformation layer.
9. dbt_project.yml is the orchestration boundary for the SQL transformations. It defines the transformation project structure and governs how the Bronze/Silver/Gold models are built in the warehouse or Databricks environment.
10. The README explicitly references Databricks, dbt, Python, Monte Carlo simulation, VaR/CVaR modeling, and efficient frontier optimization. Architecturally, that means the warehouse/lakehouse handles the transformation layers, while Python handles simulation-heavy analytics.
11. A useful way to diagram this repository is as two cooperating subsystems:
- dbt-managed data preparation and analytics modeling in SQL.
- Python-based simulation and result generation.
The handoff point is the Gold-layer input tables/files.
12. There are no clear signs of a web UI, API service, or streaming subsystem in the repository. The primary boundary is analytical compute, not application serving.
13. In short, this repo is an offline or batch-oriented risk analytics pipeline: ingest prices, transform them into statistical features, simulate portfolio outcomes, and publish risk/optimization outputs.
Main entry points: silver_daily_returns.sql Daily returns · monte_carlo_engine.py Monte Carlo · monte_carlo_simulation_results.csv Results CSV · bronze_market_prices.sql Market prices
Components 15
Orchestration
dbt project configuration and source definitions that organize the SQL pipeline.
- dbt project · Defines the dbt transformation project structure and build behavior.
dbt_project.yml - Sources · Declares upstream warehouse tables used by the Bronze layer.
source.yml
Bronze
Raw market price staging and normalization boundary.
- Market prices · Stages raw market price data before analytics transformations.
bronze_market_prices.sql
Silver
Intermediate statistical feature models derived from prices.
- Daily returns · Derives asset return series from normalized price data.
silver_daily_returns.sql - Rolling vol · Computes time-window volatility features from returns.
silver_rolling_volatility.sql - Correlation base · Builds the correlation structure used for covariance and risk analysis.
silver_correlation_base.sql
Gold
Decision-oriented risk and optimization outputs.
- Covariance input · Assembles covariance-ready inputs from Silver statistics.
gold_covariance_input.sql - MC inputs · Prepares curated inputs consumed by the Monte Carlo engine.
gold_monte_carlo_inputs.sql - Risk metrics · Computes portfolio-level risk measures for reporting.
gold_portfolio_risk_metrics.sql - VaR summary · Summarizes value-at-risk outputs from the simulation workflow.
gold_var_summary.sql - Simulation summary · Condenses Monte Carlo run results into reporting outputs.
gold_simulation_summary.sql - Efficient frontier · Derives optimization results for portfolio tradeoff analysis.
gold_efficient_frontier.sql
Python
Monte Carlo simulation runtime outside dbt.
- Monte Carlo · Executes iterative simulation logic and portfolio outcome sampling.
monte_carlo_engine.py
Artifacts
Generated outputs produced by the simulation workflow.
- Results CSV · Tabular simulation output produced by the Python engine.
monte_carlo_simulation_results.csv - Visual report · PDF visualization/report generated from simulation outputs.
monte carlo visuals.pdf
Connections 16
- dbt project → Market prices: orchestrates · dbt project builds the Bronze model.
- Sources → Market prices: feeds · Declared upstream sources supply raw price data.
- Market prices → Daily returns: transforms to · Raw prices are converted into daily returns.
- Daily returns → Rolling vol: feeds · Return series is used to compute rolling volatility.
- Daily returns → Correlation base: feeds · Return series contributes to the correlation base.
- Rolling vol → Covariance input: feeds · Volatility features inform covariance-ready inputs.
- Correlation base → Covariance input: feeds · Correlation structure supports covariance assembly.
- Covariance input → MC inputs: prepares · Covariance inputs are curated for simulation.
- MC inputs → Monte Carlo: consumed by · Python simulation consumes Gold-layer inputs.
- Monte Carlo → Results CSV: produces · Engine writes tabular simulation results.
- Results CSV → Simulation summary: summarized by · Simulation outputs are aggregated into summary metrics.
- Results CSV → VaR summary: summarized by · Simulation outputs are used to derive VaR summary.
- Simulation summary → Risk metrics: feeds · Simulation summaries contribute to risk metrics.
- VaR summary → Risk metrics: feeds · VaR summary informs portfolio risk measures.
- Risk metrics → Efficient frontier: supports · Risk metrics support portfolio optimization.
- Monte Carlo → Visual report: produces · Engine generates the visual PDF report.