How dipanjanpathak/master_ml-mern-realestate works
Project Overview and Types
• This project is a full-stack application with a strong focus on machine learning (ML) for real estate price prediction and property recommendations.
• It uses a mix of both the MERN (MongoDB, Express/Node.js, React, Node.js server) and Django stacks for different microservices. The ML workflows and model experiments are developed in Jupyter notebooks (inside ML_Operations) and further integrated into production through Django.
• The system incorporates several distinct layers: data science/ML processing, a backend API layer (with both Node.js and Django components), and a React-based frontend.
Main Components of the System
a. ML_Operations:
• Contains multiple Jupyter notebooks (data treatment, EDA, model selection, building, recommendation system) and raw/processed datasets.
• Represents the data science and model development phase where various ML models (XGBoost Regressor) are experimented and validated.
• Includes data artifacts (e.g., .csv, .pkl files) needed for training and prediction.
b. Backend Services:
• Node.js Based API:
• Found in Website/Backend, with controllers handling requests for prediction and recommendation data, routing modules that define endpoints, and server configuration.
• This service likely handles tasks such as processing user input, caching predictions, and communicating with the database (MongoDB).
• Django Based API:
• Within the Website/ml directory, Django manages API endpoints that specifically interact with the ML models (loaded either from pre-trained states or wrapped in Django apps).
• It handles ML-specific tasks such as prediction, recommendation logic, and perhaps data validation.
• Database:
• MongoDB – serving as the primary storage solution for property listings and possibly caching user query results or predictions.
c. Frontend:
• Developed in React (under Website/frontend), including a comprehensive folder structure for components, pages, static assets, and Redux Toolkit (RTK) slices for state management.
• It interacts heavily with both the backend APIs (Django and Node.js) to post user queries, receive predicted prices, and display property details.
d. Integration and Communication Flow:
• In-memory caching in the backend for quick retrieval of previously computed predictions.
• Usage of RESTful APIs for communication between the frontend and the different microservices (Node.js service for recommendations and Django service for ML predictions).
• Data flows end-to-end—from user input at the frontend to model execution in the Django service, then recommendation processing via Node.js, finally delivered from the database.