How eddaoudi-mohamed/predict_price_car works
Project type and purpose
• This is an end-to-end machine-learning pipeline to predict car prices. It includes data acquisition (scraping), data cleaning, exploratory analysis, preprocessing, model training & evaluation, and a deployment layer (API or web app).
File structure
• Top-level artifacts:
• Data.tar.zip & Models.tar.zip (archived raw/processed data and trained models)
• config/ (YAML files for split & training parameters)
• makefile (orchestration: archive, extract, clean)
• notebooks/ (Jupyter notebooks for each step in the ML workflow)
• Documentation/ (reports, graphs, tasks)
• requirements.txt
• Absence of src/ or app/ directories in the tree; they are implied in README.
• The pipeline is scripted via notebooks and governed by config files and Makefile targets.
Key README insights
• Seven sequential stages: Scraping → Cleaning → EDA → Preprocessing → Training → Evaluation → Deploying
• Dependencies: pandas, numpy, scikit-learn, matplotlib, seaborn, Flask/Django
• Makefile targets automate data/model archive and extraction.
a. Main components
1. External Data Source (Websites)
2. Scraper Module (notebooks/ScrapData.ipynb or src/scraper.py)
3. Raw Data Store (folder extracted from Data.tar.zip)
4. Cleaning Module (notebooks/CleaningData.ipynb)
5. EDA Module (notebooks/EDA.ipynb)
6. Preprocessing Module (notebooks/Preprocessing.ipynb + config/split_config.yaml)
7. Training Module (notebooks/ModelTraining.ipynb + config/training_config.yaml)
8. Model Registry (Models folder, archived in Models.tar.zip)
9. Evaluation Module (notebooks/Evaluation.ipynb)
10. Deployment Layer (app/, Flask/Django API or web UI)
11. Orchestration & Configuration (Makefile, requirements.txt)
b. Relationships & data flow
• Raw Data Store → Cleaning → Cleaned Data Store
• Cleaned Data Store → EDA (for insights, reports in Documentation/Graphs)
• Cleaned Data → Preprocessing → Train/Test Splits (governed by split_config.yaml)
• Splits → Training → Candidate Models → Model Registry
• Candidate Models + Test Splits → Evaluation → Metrics
• Selected Model → Deployment Layer → API Endpoint
• Orchestration (Makefile) interacts with Data Store & Model Registry
c. Architectural patterns & principles
• Pipeline pattern: sequential modules transforming data
• Configuration-as-code: all parameters in YAML
• Versioned artifacts: data & models zipped for reproducibility
• Modular notebooks for each stage (can be refactored to Python modules for production)
d. Technologies & frameworks
• Python, Jupyter Notebooks
• pandas, numpy, scikit-learn
• matplotlib, seaborn (visualization)
• Flask or Django for API deployment
• Makefile for orchestration
• YAML for configuration