How cristian-pulido/uncertainty-informed-decision-making works
Project Context & Overall Components
• Project Type: This repository is an open-source analytical and experimental tool, centering on predictive modeling under uncertainty. It combines both production-grade modules (in the src folder) and research/experimental outputs (in the articulos folder).
• Purpose: To provide code, models, and tools (such as Conformal Prediction) to inform decision-making when dataset limitations or intervention uncertainties exist.
Main Components
a. Data Layer & External Data:
• External Data Setup: As noted in the README, large simulation outputs and sensitive real-world datasets exist outside the git repository in “uncertainty-informed-data”. Mark this as an external dependency that feeds into the system.
• Data Sources: Internal CSV files (e.g., in articulos/uncertainty-hotspots/data) and simulated/synthetic data generation (src/data_generation/hotspots.py).
b. Data Generation Module:
• Located under src/data_generation.
• Responsible for generating synthetic or simulation data used in analysis. This feeds into model training and evaluation.
c. Modeling Components:
• Found under src/models and the results folder in articulos. This includes various models such as naive_cell, poisson_cell, poisson models, and dummy models.
• Separate carefully between:
• Model definitions (code in src/models).
• Model outputs and artifacts (stored in articulos/uncertainty-hotspots/results/models).
d. Conformal Prediction & Uncertainty Quantification:
• Under src/conformal, the module (e.g., mapie_per_cell.py) handles the core work of applying Conformal Prediction methods.
• This is key to the project and should be highlighted as a specialized layer that integrates with the model training and evaluation pipeline.
e. Evaluation Suite:
• Located in src/evaluation. This includes scripts for evaluating models (via CP, spatial metrics, and temporal evaluation).
• Demonstrates the metric calculation and testing framework used to assess decision-making under uncertainty.
f. Utility Functions:
• Found under src/utils. Includes support for data partitioning, spatial processing, and visualization.
• These facilitate handling data flows and producing outputs used in reports and notebooks.
g. Experimental / Demonstration Notebooks:
• In articulos/uncertainty-hotspots/notebooks, these provide interactive experiments that guide users through:
• Synthetic data tests.
• Model training and evaluation.
• Visualization and post-processing of model results.
• This layer serves as the research or demonstration interface for the underlying modules.
Relationships and Data Flow
• Data Generation outputs are used by the Modeling components.
• Conformal Prediction integrates with the Modeling phase to estimate uncertainty.
• The Evaluation module consumes outputs from the Modeling and Conformal Prediction layers to produce performance metrics.
• Utility modules support data partitioning, spatial transformations, and visualization across many stages.
• The experimental notebooks orchestrate these modules, providing a guided pipeline and visualization of results.
Architectural Patterns and Design Details
• Modularity: A clear separation between data ingestion, processing (data generation, modeling, prediction), and evaluation.
• Extensibility: The use of separate directories (e.g., src/models, src/evaluation) suggests that the design should allow plugging in additional models or evaluation criteria.
• Iterative Research Workflow: The presence of notebooks indicates an experimental, iterative approach where results feed back into model improvements.