How mcallaghan/regional-impacts-map works
Main Components
• Data Ingestion & Preprocessing:
• Literature Identification: “literature_identification” contains scripts and notebooks to gather, filter, and process bibliographic and study metadata.
• Supplementary Data: “supplementary_data” and “tables” hold raw/aggregated data that feed into the analysis.
• Analysis & Machine Learning Module:
• Analysis Directory: “analysis” is the core where numerous Jupyter notebooks and Python scripts orchestrate clustering, cross-validation, model selection, and prediction. In particular, the “2-cluster-ml-scripts” folder contains scripts to run prediction pipelines (including SVM and BERT models) and perform extensive model evaluations.
• Detection and Attribution: “detection_and_attribution” contains notebooks and scripts dedicated to attributing climate trends (temperature and precipitation) to underlying patterns.
• Visualization & Output Generation:
• Figures/Plots: “figures”, “plots”, and “presentation” directories store the final outputs – PDFs, images, and LaTeX files – that represent the scientific results (including interactive map links referenced in the README).
• Manuscript Generation: “manuscript” holds the LaTeX source and PDFs for the scientific paper and supplementary materials.
Relationships & Interactions
• Data & Input Flow:
• Raw data (from literature and supplementary data) are processed using scripts in “literature_identification” and “analysis”.
• Modeling & Analysis Flow:
• Preprocessed inputs feed into the ML pipelines in “analysis” (cluster-ml-scripts) and into the “detection_and_attribution” module.
• The outputs (predictions, cross-validation results, error logs) are then aggregated.
• Visualization & Reporting Flow:
• The aggregated results drive the generation of figures (in “figures” and “plots”), which in turn support the manuscript and presentation documents.
• The interactive map mentioned in the README is an additional presentation layer accessible via a web link.
Architectural Patterns
• Modular Pipeline:
• Each directory represents a distinct stage of the research pipeline: data collection (literature_identification), ML analysis (analysis and detection_and_attribution), and results presentation (figures, plots, manuscript, presentation).
• Separation of Concerns:
• Different modules are strictly separated, allowing independent development or rerunning of model selection, cross-validation, and visualization scripts.
• Data Flow Orientation:
• The system is architected as a sequential flow: starting from data ingestion, through complex computation in ML pipelines, to ultimately producing publication-quality outputs.
• Use of Notebooks and Scripts:
Relevant Technologies/Frameworks
• Python is the primary language, with extensive Jupyter notebook usage.
• Machine learning techniques using classification models (SVM, BERT) are central.
• Data visualization tools (likely matplotlib, LaTeX for PDF generation) are used in producing figures and the manuscript.
• Version control (Git) is in place, as indicated by .gitignore and LICENSE.txt.