How davidecallegaro/big_data_in_finance_2020 works
Project type and purpose
• This is a data-science / network-simulation project implemented entirely in Jupyter notebooks.
• Purpose: build a financial network from bilateral claims and credit‐rating data, then run an SIR (Susceptible‐Infected‐Recovered) style default‐cascade simulation to understand the spread of systemic risk.
File structure
• Top‐level notebooks:
• BDF – First Submission.ipynb, BDF – Final Notebook.ipynb, Data Exploration.ipynb
• “Big Data in Finance – Presentation.pdf” and other static deliverables
• Data directory (data/_bin and data/comtrade): raw CSV/XLSX exports of reserves, GDP series, investment, trade (COMTRADE), credit scores, foreign claims
• README.md describes the end-to-end workflow
• No application code beyond analysis notebooks and raw data
Key insights from the README
• Four major steps:
1) Build the network from bilateral claims
2) Model parameters (β from claims, γ from credit ratings)
3) Run per‐country default‐cascade simulations
4) And compare epidemic curves and derive policy insights
a. Main components
1. Data Sources
– BIS foreign claims CSVs (data/_bin)
– COMTRADE bilateral trade CSVs (data/comtrade)
– Credit rating/credit_score.csv
– Macroeconomic indicators (GDP, reserves)
2. Data Ingestion & Cleaning
– “Data Exploration.ipynb”: load, clean, standardize CSV/Excel
– Outputs: cleaned data tables
3. Network Builder
– Code cells in notebooks that take cleaned bilateral claims → weighted graph edges
4. Parameter Calculator
– Compute β (transmission rate) from normalized claims
– Compute γ (recovery rate) from credit scores
5. Simulation Engine
– SIR module in BDF notebooks: iteratively propagate defaults through network
– Iterates simulation per country origin
6. Results Aggregator & Visualizer
– Generate time‐series of Susceptible/Infected/Recovered counts
– Produce plots, summary tables
7. Reporting & Presentation
– Final notebook compiles figures and conclusions
– Exports to PDF (Big Data in Finance – Presentation.pdf)
b. Relationships & interactions
– Annotation: “Iterate simulation for each origin country” between Simulation Engine and Results Aggregator
c. Architectural patterns & design principles
– Modular pipeline: separate data ingestion, preprocessing, simulation, and reporting
– Notebook‐centric “orchestration” rather than script/function libraries
– Emphasis on reproducibility: raw data → cleaned data → simulation → plots
d. Key technologies & libraries
– Python, Jupyter notebooks
– pandas (CSV/Excel I/O, dataframes)
– networkx (graph construction)
– numpy (numerical computation)
– matplotlib / seaborn (plotting)
– comtrade API or local CSVs for trade data