How kunaly0/credit_risk_projects works
This repository builds a traceable credit-risk data foundation: Freddie Mac loan files and FRED macroeconomic series are transformed and loaded into PostgreSQL, where quality checks and audit records support downstream reporting. The graph includes the documented schema, SQL quality rules, summary views, and Power BI dashboard. Source excerpts confirm the loader, quality-suite, macro-loader, and checksum behavior; other subsystem links are included only where the README documents them.
Main entry points: load_fred.py Macro loader · loader.py Loan ETL loader · run_dq_suite.py DQ suite runner · verify_raw_checksums.py Checksum verifier
Components 16
Loan ingestion
- Sample load runner
scripts/load_sample.py - Loan ETL loader
src/etl/loader.py - Loan transforms
src/etl/transforms.py - Star schema setup
sql/01_create_dimensions.sql
Macro data
- Macro loader
src/macro/load_fred.py - Macro settings
config/macro_config.yaml
Quality and governance
- DQ suite runner
src/dq/run_dq_suite.py - DQ rule queries
sql/dq - Checksum verifier
scripts/verify_raw_checksums.py - Source checksums
docs/provenance
Reporting
- Dashboard views
sql/12_create_dashboard_views.sql - Power BI dashboard
dashboards/project0_dq.pbix
Other components
- Freddie Mac files
- PostgreSQL
- FRED API
- Report user
Connections 19
- Freddie Mac files → Sample load runner: provides files
README.md - Sample load runner → Loan ETL loader: dispatches loads
scripts/load_sample.py - Loan ETL loader → Loan transforms: uses transforms
src/etl/loader.py - Loan ETL loader → PostgreSQL: loads loan rows
src/etl/loader.py - Loan ETL loader → PostgreSQL: records load audit
src/etl/loader.py - Star schema setup → PostgreSQL: creates schema
README.md - FRED API → Macro loader: provides series
README.md - Macro settings → Macro loader: configures series
src/macro/load_fred.py - Macro loader → PostgreSQL: reads loan states
src/macro/load_fred.py - Macro loader → PostgreSQL: replaces macro tables
src/macro/load_fred.py - Macro loader → FRED API: downloads series
src/macro/load_fred.py - DQ suite runner → PostgreSQL: queries data
src/dq/run_dq_suite.py - DQ suite runner → PostgreSQL: writes DQ results
src/dq/run_dq_suite.py - DQ suite runner → DQ rule queries: implements checks
README.md - Checksum verifier → Source checksums: reads records
scripts/verify_raw_checksums.py - Checksum verifier → FRED API: checks saved pulls
scripts/verify_raw_checksums.py - Dashboard views → PostgreSQL: queries summaries
README.md - Power BI dashboard → Dashboard views: uses views
README.md - Report user → Power BI dashboard: views dashboard