How lahsiva/whatsapp-chat-analyser works
Project type & purpose (from README + tree)
This is a single-process, local (or single-node) Streamlit web app for WhatsApp chat analytics:
• Input: user-uploaded exported WhatsApp chat text file
• Processing: parsing + preprocessing + NLP feature extraction (TF‑IDF), sentiment classification (Naive Bayes), lexicon-based sentiment
• Output: interactive visualizations (timelines, activity maps, word clouds, top terms/topics, sentiment/user activity insights)
There’s no evidence of a separate backend service, database, or microservices; the “backend” is simply the Python Streamlit runtime executing analysis in-memory.
File structure analysis → architectural components
Top-level files imply a simple module-based architecture:
• app.py / main.py
• Likely the Streamlit entrypoint(s): UI layout, file uploader, user controls (filters/user selection), orchestration of analysis calls, and rendering charts.
• For diagramming, treat this as the Presentation + Orchestration layer.
• preprocessor.py
• Core parsing/cleaning: converting raw WhatsApp export text into structured data (likely a pandas DataFrame with columns like datetime, user, message).
• Treat this as the Ingestion/Preprocessing layer.
• helper.py
• “Business logic” / analytics functions: timelines, activity maps, word clouds, TF-IDF vectorization, Naive Bayes inference/training (or loading), lexicon scoring, aggregations.
• Treat this as the Analytics / Feature Engineering / Visualization Data Prep layer.
• stop_hinglish.txt
• A domain-specific stopword list used in tokenization/wordcloud/TF‑IDF cleanup.
• Treat as a Static resource feeding NLP.
• setup.sh
• Environment bootstrap (dependency install, run commands).
• Treat as Build/Run tooling (not runtime).
• README.md
• High-level purpose; no additional runtime components implied.
At runtime the flow is essentially:
helper.py computes
• aggregations (timelines, activity heatmaps)
• NLP artifacts (tokenization, stopword removal using stop_hinglish.txt)
• TF‑IDF features
• sentiment via Naive Bayes + lexicon scoring
System Context (Level 0) – who uses it and what external artifacts exist
This gives maximum clarity without inventing non-existent services.
Ingestion & Preprocessing
• preprocessor.py
• Parsing WhatsApp raw text
• Normalizing timestamps/users/messages
• Producing structured dataset (e.g., DataFrame)
Analytics / NLP / Insights
• helper.py
• Timeline generation (daily/monthly)
• Activity map (hour/day heatmap)
• Word frequency / top words
• Word cloud generation
• Sentiment analysis:
• TF‑IDF vectorizer
• Naive Bayes classifier
• Lexicon-based sentiment scorer
Static Data / Configuration
• stop_hinglish.txt → used by helper NLP pipeline (stopword filtering)
Tooling (out-of-band)
• app.py/main.py → preprocessor.py: “parse(uploaded_chat)”
• preprocessor.py → app.py/main.py: “structured_messages_df”
• app.py/main.py → helper.py: “compute_metrics(df, filters)”
• helper.py → stop_hinglish.txt: “load/consume stopwords”
• helper.py → app.py/main.py: “metrics + chart-ready datasets + wordcloud image”
• app.py/main.py → Browser: “render plots/tables”
Architectural patterns & design principles to highlight (accurately)
• Layering by responsibility:
• UI/orchestration (Streamlit entry)
• preprocessing (pure transformation)
• analytics helpers (computations)
• Pipeline dataflow:
• raw text → structured dataset → features → insights → visualization
• Stateless-by-default (likely):
• results recomputed per upload/session (unless caching is used)
• In-memory analytics:
• no persistence layer implied