How charankumarkharade/dax40-sentiment-monitor works
This repository describes a DAX 40 news-monitoring pipeline: it collects Tagesschau articles, cleans and enriches them with sentiment and company attribution, models results for analytics, and supports alerts and BI. Notebook and SQL contents were not sampled, so their internal wiring is shown only where the README documents it; SQL capabilities are included without inferred data-flow links.
Main entry points: Polished_Gold.ipynb Gold enrichment · Cleaning_Silver.ipynb Silver cleaning · Star_models.ipynb Star modeling · Ingestion_Bronze.ipynb Bronze ingestion
Components 21
Ingestion and orchestration
- Fabric pipeline
- Bronze ingestion
notebooks/Ingestion_Bronze.ipynb
Medallion data stages
- Bronze archive
- Silver cleaning
notebooks/Cleaning_Silver.ipynb - Silver articles
- Gold enrichment
notebooks/Polished_Gold.ipynb - Company aliases
config/dax40_companies.py - Gold analytics
Analytics and risk
- Star modeling
notebooks/Star_models.ipynb - Star schema
- Reputation risk
sql/01_reputation_risk_matrix.sql - Sentiment trends
sql/02_sentiment_trend.sql - Sentiment breakdown
sql/03_sentiment_breakdown.sql - Company breakdown
sql/04_company_daily_breakdown.sql - Breaking news alert
sql/05_breaking_news_alert.sql - Negative spike detection
sql/06_negative_spike_detection.sql
BI and alerting
- Power BI
- Office 365 email
Other components
- Risk manager
- Tagesschau API
- German BERT
Connections 19
- Fabric pipeline → Bronze ingestion: dispatches
- Fabric pipeline → Silver cleaning: dispatches
- Fabric pipeline → Gold enrichment: dispatches
- Fabric pipeline → Star modeling: dispatches
- Tagesschau API → Bronze ingestion: provides articles
- Bronze ingestion → Bronze archive: writes raw JSON
- Silver cleaning → Bronze archive: reads archive
- Silver cleaning → Silver articles: writes cleaned articles
- Gold enrichment → Silver articles: reads articles
- Gold enrichment → Company aliases: uses aliases
- Gold enrichment → German BERT: scores sentiment
- Gold enrichment → Gold analytics: writes analytics
- Star modeling → Gold analytics: reads analytics
- Star modeling → Star schema: writes dimensions
- Star schema → Power BI: serves model
- Gold enrichment → Fabric pipeline: returns alert text
- Fabric pipeline → Office 365 email: sends notifications
- Fabric pipeline → Risk manager: notifies failures
- Office 365 email → Risk manager: delivers alerts