How sthirani/sentiment-analysis works
Main Components
a. Twitter Data Collection:
• Twitter API as an external service that the system interacts with.
• A component (labeled “Twitter.py”) that handles authentication and fetching tweets using the Twitter API.
b. Data Cleaning and Preprocessing:
• “CleanData.py” module which preprocesses raw tweet data by removing special characters, stopwords, and punctuation.
• Optionally, indicate “DisableSSL.py” as a utility used to manage SSL issues during resource downloads (such as NLTK modules).
c. Sentiment Analysis:
• “SentimentData.py” module where the sentiment analysis takes place using NLTK’s VADER.
• Use of NLP techniques and possible tokenization and cleaning performed before analysis.
d. Data Persistence & Storage:
• CSV files (e.g., “clean_sentences_nike.csv”) where the cleaned data and perhaps analysis results are stored.
e. Visualization:
• Generation of word clouds with the WordCloud module as a separate step after analysis.
f. Dependencies & Libraries:
Relationships and Interactions
• Process flow from Twitter.py –> CleanData.py –> SentimentData.py as well as a separate branch towards visualization (WordCloud generation).
• That the data flows into CSV storage (clean_sentences_nike.csv) after cleaning, either as an intermediate step or output of the pipeline.
Architectural Patterns & Design Principles
• Pipeline nature of the design (data ingestion – preprocessing – analysis – visualization).
• Separation of concerns: one module handles data collection (Twitter.py), another is responsible for cleaning (CleanData.py), and a third manages analysis (SentimentData.py).
• That this is essentially a single-service, script-based pipeline rather than a distributed or microservices architecture.
Details
• Major functional modules and any utility scripts, even if they appear as support (e.g., DisableSSL.py), to fully document the architecture.