How matth-info/damaged-building-detector works
Project type and purpose
• This is an end-to-end machine-learning (computer-vision) pipeline for “before/after” satellite image analysis, specifically detecting building damage.
• It’s organized with preprocessing, training, inference, and deployment (via Docker/MLOps) phases.
File structure
• Top-level directories:
• configs/ ← YAML files for model, logging, etc.
• data/ ← raw and sample imagery datasets
• notebooks/ ← exploratory data analysis & prototyping
• scripts/ ← shell/Python wrappers for training & testing
• src/ ← core Python modules subdivided into:
– datasets/ (data‐loading & augmentation)
– models/ (network definitions)
– losses/ (loss functions)
– metrics/ (evaluation metrics)
– training/ (training loop, augmentations)
– inference/ (prediction API)
– explainability/ (Grad-CAM)
– utils/ (visualization, helpers)
• tests/ ← unit tests for modules
• Dockerfile, docker-compose.yaml, Makefile, .githubignore
README for additional insights
• Confirms:
• “Satellite Image Damage Detection Pipeline”
• MLOps with Docker, reproducible workflows
• Steps: preprocess, train, infer
1. Data Storage
– Raw imagery buckets or local folders (data/)
– Sample data (data/data_samples/)
2. Configuration
– configs/config.yaml, logging.yaml
3. Data Preprocessing
– src/datasets/ + preprocessing scripts (scripts/run_*.sh)
4. Model Training
– src/training/ + src/models/, src/losses/, src/metrics/
– Entrypoint: scripts/run_training.sh
5. Model Artifacts
– Checkpoint storage (e.g., volumes or cloud storage)
6. Inference Service
– src/inference/predict.py + functional API
– Dockerized REST or CLI service
7. Explainability Module
– src/explainability/grad_cam.py
8. MLOps & Deployment
– Dockerfile, docker-compose.yaml, Makefile
– CI/CD pipelines (not shown in code but implied)
9. Monitoring & Logging
– logging.yaml → centralized logs
• Data Storage → Data Preprocessing
• Configurations → Preprocessing, Training, Inference
• Preprocessing → Training
• Training → Model Artifacts
• Model Artifacts → Inference
• Inference → Prediction Output
• Explainability module can tap into both Training (for validation) and Inference (for live explanations)
• All components run inside Docker container(s)
c. Architectural patterns and design principles
• Modular separation of concerns: Data → Compute → Serving
• Pipeline orchestration via scripts + Docker (loosely coupled stages)
• Config-driven: single source of truth in YAML
• Reproducibility: notebooks + Docker for dev, scripts for prod
• Test-driven development: tests/ validating each module
d. Key technologies
• Python 3.8+
• PyTorch or TensorFlow (notebooks indicate both)
• Docker & Docker Compose for containerization
• YAML-based configs
• Jupyter notebooks for prototyping
• Standard ML libraries: numpy, pandas, rasterio, albumentations
Essential details without overcomplicating
• External dependencies (e.g., cloud storage, if used) only if relevant.