How deshawa77/world-pulse-research works
World Pulse is a global and country-level crisis-intelligence product: it gathers public, economic, environmental, and market signals, derives risk and forecasts, then exposes results through APIs and an operator dashboard. The principal workflow runs from collectors through streaming/orchestration and feature analysis to persistence and serving; separate branches support country intelligence, scenarios, model governance, and monitoring. The source excerpts and index support several backend relationships, but the README is empty and frontend-to-backend/API details are not shown, so those links are omitted or marked unverified.
Main entry points: orchestrator.py Pipeline orchestrator · mongo.py MongoDB store · kafka_client.py Kafka transport · main.py API and intelligence service
Components 27
Signal ingestion
- Pipeline orchestrator
orchestrator.py - Signal collectors
collectors - Crypto collector
collectors/coingecko.py - Weather collector
collectors/weather.py - Health collector
collectors/who.py - Trend collector
collectors/trends.py - Kafka transport
backend/kafka_client.py
Risk intelligence
- Data preprocessing
processing/preprocess_data.py - Topic intelligence
processing/topic_modeling_with_nlp.py - Robust topic pipeline
processing/robust_topic_pipeline.py - Feature builder
processing/daily_feature_builder.py - Global risk
processing/global_risk.py - Crisis detector
processing/global_crisis_detector.py - Country risk
processing/country_daily_risk.py - Causal explanation
processing/causal_risk_navigator.py - Scenario analysis
processing/counterfactual_engine.py
Serving and storage
- API and intelligence service
backend/main.py - MongoDB store
database/mongo.py - Feature store
feature_store/feature_store.py
Operator experience
- Web application
world-pulse-frontend/src/App.tsx - Frontend API client
world-pulse-frontend/src/services/api.ts - Intelligence dashboard
world-pulse-frontend/src/pages/Dashboard.tsx - Prediction views
world-pulse-frontend/src/pages/TrendPrediction.tsx
Models and operations
- Model registry
feature_store/model_registry.py - Scheduled jobs
scheduler/runner.py - Pipeline observability
backend/observability.py
Other components
- Operator
Connections 31
- Pipeline orchestrator → Signal collectors: runs collectors
orchestrator.py - Pipeline orchestrator → Kafka transport: starts stream
orchestrator.py - Pipeline orchestrator → Data preprocessing: calls preprocessing
orchestrator.py - Pipeline orchestrator → Topic intelligence: runs modeling
orchestrator.py - Pipeline orchestrator → Feature builder: builds features
orchestrator.py - Pipeline orchestrator → Global risk: runs risk analysis
orchestrator.py - Pipeline orchestrator → Feature store: writes features
orchestrator.py - Pipeline orchestrator → Pipeline observability: reports events
orchestrator.py - Crypto collector → Kafka transport: publishes signals
collectors/coingecko.py - Crypto collector → MongoDB store: stores records
collectors/coingecko.py - Weather collector → Kafka transport: publishes signals
collectors/weather.py - Weather collector → MongoDB store: stores records
collectors/weather.py - Health collector → Kafka transport: publishes signals
collectors/who.py - Health collector → MongoDB store: stores records
collectors/who.py - Trend collector → Kafka transport: publishes signals
collectors/trends.py - Trend collector → MongoDB store: stores records
collectors/trends.py - Feature builder → MongoDB store: reads data
processing/daily_feature_builder.py - Feature builder → Global risk: uses risk features
processing/daily_feature_builder.py - Crisis detector → MongoDB store: reads records
processing/global_crisis_detector.py - Scenario analysis → Causal explanation: gets explanation
processing/counterfactual_engine.py - API and intelligence service → Country risk: serves country risk
backend/main.py - API and intelligence service → Global risk: serves global risk
backend/main.py - API and intelligence service → MongoDB store: reads and writes
backend/main.py - Web application → Intelligence dashboard: routes to
world-pulse-frontend/src/App.tsx - Web application → Prediction views: routes to
world-pulse-frontend/src/App.tsx - Intelligence dashboard → Frontend API client: uses client
- Prediction views → Frontend API client: uses client
- Frontend API client → API and intelligence service: requests data
- Scheduled jobs → Country risk: runs validation
scheduler/runner.py - Scheduled jobs → Model registry: manages models
- Operator → Web application: uses