How kajamalan15/airline-sentiment-analysis-nlp works
Purpose and Entry Points
This repository is a Python NLP project for classifying airline-related tweets into positive, neutral, or negative sentiment. The interactive exploration/training workflow is centered on Tweets.ipynb; the application-facing entry point is likely app.py.
Source Data and Training Corpus
The primary labeled dataset is Tweets (1).csv. It supplies tweet text and sentiment labels for preprocessing, exploratory analysis, TF-IDF feature generation, model training, and evaluation. The project is batch-oriented around this local CSV rather than a connected streaming or API ingestion system.
NLP Preprocessing and Feature Engineering
Tweet text is cleaned before modeling, including removal of mentions, hashtags, URLs, and stopwords as described by the project documentation. The transformed text is converted into sparse TF-IDF vectors. The fitted vectorizer is persisted as tfidf_vectorizer.pkl, creating a compatibility boundary: inference text must undergo the same preparation and vectorization assumptions used during training.
Model Training and Selection
The notebook workflow in Tweets.ipynb develops sentiment classifiers using Naïve Bayes and Random Forest approaches, compares their results, and selects a best-performing model. The selected trained estimator is serialized in best_sentiment_model.pkl; this file is the central model artifact consumed by inference.
Inference Application Boundary
app.py is the deployment-oriented integration point. It is expected to load best_sentiment_model.pkl and tfidf_vectorizer.pkl, accept tweet input, convert it to the model feature space, and return a sentiment class. This separates interactive training work from reusable prediction-time artifacts.
Analysis and Visualization
Tweets.ipynb also serves as the exploratory and reporting surface for the dataset and classification outcomes. Matplotlib and Seaborn are the intended visualization stack, supporting sentiment distribution and model-result analysis rather than being part of the core runtime prediction path.
Runtime and Supporting Assets
The project runs on Python with dependencies declared in requirements.txt, including the data-science, NLP, and visualization ecosystem described in the README. spaCy is a material NLP dependency, while scikit-learn supplies TF-IDF and classifier implementations. .devcontainer/devcontainer.json provides a reproducible development-environment definition. ImageFolder/Images and Project.mp4 appear to be presentation/demo assets, not production model inputs.
Main entry points: app.py Inference application · Tweets.ipynb Training notebook · requirements.txt Python dependencies · tfidf_vectorizer.pkl Fitted vectorizer
Components 15
Training & Analysis
Interactive batch workflow.
- Labeled tweets · Tweet text and sentiment labels.
Tweets (1).csv - Training notebook · Explores data, trains, compares, and reports models.
Tweets.ipynb - Text preprocessing · Removes tweet noise and stopwords.
- TF-IDF features · Converts cleaned tweets to sparse vectors.
- Model selection · Compares Naïve Bayes and Random Forest classifiers.
- Fitted vectorizer · Defines the inference feature-space contract.
tfidf_vectorizer.pkl - Selected sentiment model · Best-performing trained estimator.
best_sentiment_model.pkl - Analysis visualizations · Sentiment and model-result plots.
Prediction Runtime
Application-facing inference path.
- Inference application · Accepts tweet input and returns a sentiment class.
app.py - Tweet input
- Inference preparation · Must preserve training preprocessing assumptions.
- Sentiment prediction · Positive, neutral, or negative.
Other components
- Python dependencies · Includes spaCy, scikit-learn, Matplotlib, and Seaborn.
requirements.txt - Development environment · Reproducible development setup.
.devcontainer/devcontainer.json - Presentation assets · Non-production images and video.
ImageFolder/Images
Connections 15
- Labeled tweets → Training notebook: explored and labeled for training
- Training notebook → Text preprocessing: cleans tweets
- Text preprocessing → TF-IDF features: feeds cleaned text
- TF-IDF features → Model selection: training features
- Model selection → Selected sentiment model: serializes best model
- TF-IDF features → Fitted vectorizer: persists fitted vectorizer
- Training notebook → Analysis visualizations: produces analysis
- Tweet input → Inference application: submitted to
- Inference application → Inference preparation: prepares input
- Fitted vectorizer → Inference application: loads feature contract
- Selected sentiment model → Inference application: loads estimator
- Inference preparation → Sentiment prediction: classifies with loaded artifacts
- Python dependencies → Training notebook: supports training stack
- Python dependencies → Inference application: supports runtime stack
- Development environment → Python dependencies: provisions environment