How yashspr/sentiment_analysis_ml_part works
Project type and purpose
• This is a full-stack sentiment-analysis application composed of two subprojects:
a) “sentiment_analysis_ml_part” – a Python-based ML pipeline and Flask microservice that trains and serves a sentiment-analysis model.
b) “web_sentiment_analysis” – a Node.js frontend/back-end web application that provides a UI, forwards user input to the ML service, and persists results in MongoDB.
• The Python part handles data ingestion (CSV), preprocessing, feature extraction, model training, and model serving via a REST API.
• The Node.js part hosts the client interface, calls the Flask API for sentiment predictions, and stores or displays results.
File structure (Python ML part)
• Top-level scripts:
• preprocess.py – cleans and tokenizes raw text.
• feature_extraction.py & ft.py – extract features (e.g., FastText embeddings).
• train.py & classification.py – train and evaluate the model.
• main.py – likely a convenience script combining preprocess/feature/train steps.
• server.py – Flask app exposing /predict endpoints.
• Data directory “csv_files/” – raw CSVs and training data.
• environment.yml / requirements.txt – dependencies (spaCy, FastText, Flask).
• No database here; persistence is in flat files and saved model artifacts.
README for additional insights
• The README confirms the two-service architecture.
• Python service uses Flask, spaCy, FastText.
• Node.js service requires MongoDB and serves the frontend.
• Interaction: Web UI → Node.js API → Python Flask → model → back to Node.js → UI/DB.
Components and relationships
a. Main components:
1. Web Browser (Client UI)
2. Node.js Web Server (Express or similar)
3. MongoDB Database
4. Python ML Service (Flask)
5. CSV Data Store & Model Artifacts
6. External Libraries (spaCy, FastText)
b. Interactions:
• Client sends sentiment query (text) → Node.js server
• Node.js server logs request to MongoDB (optional)
• Node.js forwards text via HTTP REST to Python ML service
• Python ML service:
• Runs preprocessing → feature extraction → loads trained model → classification → returns sentiment score
• Node.js receives result, stores it in MongoDB (optional), and returns response to client.
c. Architectural patterns:
• Microservices (separate Python service and Node.js service)
• Data pipeline (preprocess → feature extraction → training → model serving)
• REST API for inter-service communication
• “Train once, serve many” – training pipeline is decoupled from serving pipeline.
d. Key technologies:
• Python: Flask, spaCy (en_core_web_sm), FastText, pandas
• Node.js: Express (or similar), MongoDB driver
• MongoDB: storage for user requests and/or analytics
• Conda (environment.yml) for Python deps