How vanthienphan2004/morphological-feature-analysis-of-retinal-pigment-epithelial-cells-from-c57bl-6j-mice-during-aging works
Project type and purpose
• This is a configuration-driven Python data-analysis pipeline that ingests TIFF image crops, extracts morphological features, cleans and aggregates them, trains a stacking ensemble classifier (XGBoost, LightGBM, CatBoost + LogisticRegression meta-learner), and outputs model artifacts, reports, and plots.
File structure
• Top-level:
• config.json (runtime parameters)
• main.py (pipeline orchestrator)
• requirements.txt
• analysis_results/ (generated models, reports, plots)
• scripts/ (modular step implementations)
• scripts/ maps 1:1 to pipeline stages (feature_extraction, clean_and_prepare, train_and_save_pipeline, feature_insights, etc.).
• main.py imports and calls these scripts in sequence based on config.json.
README for additional insights
• Confirms a 7-step automated workflow.
• Describes each step.
• Notes input directories, output directories, CSVs, model bundles, report formats.
1) Configuration Loader (reads config.json)
2) Feature Extraction Module (scripts/feature_extraction.py)
3) Data Cleaning & Preparation Module (scripts/clean_and_prepare.py)
4) Model Training & Saving Module (scripts/train_and_save_pipeline.py)
5) Feature Importance & Insights Module (scripts/feature_insights.py)
6) Output Storage (analysis_results/ with subfolders reports/, models/, plots/)
7) Input Storage (raw TIFF folders or CSVs)
b. Relationships & data flow
c. Architectural patterns / design principles
• Modular, single-responsibility scripts for each stage
• Orchestration layer (main.py) for sequential execution based on config
• Separation of code (scripts/) vs. data (analysis_results/) vs. config (config.json)
d. Key technologies & frameworks
• Python 3.x, joblib
• XGBoost, LightGBM, CatBoost, scikit-learn (stacking ensemble)
• TIFF image processing libraries (e.g. scikit-image / PIL)
• pandas, numpy for data manipulation