How datpham0412/covid19-prediction-model works
Main components
• Data Ingestion & Migration – Represent the scripts that fetch data from external sources (scripts/fetch_data.py) and then migrate raw data into a usable form (scripts/migrate_data.py). This is where the project collects Covid‑19 case data and mobility data.
• Data Processing and Feature Engineering – Include the components for cleaning and processing the data (scripts/data_processing.py), performing exploratory data analysis (scripts/eda_visualization.py), engineering new features (scripts/feature_engineering.py), and splitting the data into training and testing sets (scripts/split_data.py).
• Model Training & Evaluation – Highlight the model lifecycle, with training (scripts/model_training.py) and evaluation (scripts/model_evaluation.py). Also note that serialized models (e.g., models/vietnam/linear_regression_model.pkl) are stored as artifacts.
• C++ Core Processing – Emphasize the components in the src folder (src/data_processor.cpp and src/main.cpp) that provide efficient data processing capabilities along with integration to SQLite (through the included SQLiteCpp library). This module, built via CMake (CMakeLists.txt) and linked with external libraries, underpins performance-critical operations.
• Visualization & Interpretation – Represent the interactive analysis and reporting layer using Jupyter Notebooks (notebooks/interpret_predictions.ipynb) which enable detailed data exploration and visual interpretation of model results.
Out relationships and data flow
• The prepared dataset is then used in the Model Training component, where the machine learning algorithm (built with Python libraries like scikit‑learn) learns the trends. Post-training, evaluations are carried out and results are stored.
• Additionally, show how the C++ core (compiled via CMake from src/) integrates with the system for efficient processing and SQLite database interactions. This might be represented as a side module that either feeds data to or works in tandem with the Python processing pipeline.
Architectural patterns and key design principles
• Separation of concerns – Different folders clearly separate data ingestion, processing, model training, and visualization.
• Modular design – Each script or module has a dedicated responsibility within the overall machine-learning pipeline.
• Hybrid technology stack – Python for high-level data science and visualization, C++ for performance-critical components, and SQLite for database management.
• Integration points – Explicit integration of SQLite through the SQLiteCpp library in C++ (reflected in both the “include” and “src” directories and CMake build configuration).
Relevant technologies and frameworks
• Python: Pandas, scikit‑learn, matplotlib, seaborn, dill, Joblib, and Jupyter Notebook.
• C++: Employed for performance, with Google Test for unit testing, and built using CMake.
• SQLite: Utilized via the SQLiteCpp library for handling data persistence.
• Cross-platform building: CMake to orchestrate builds across environments.