How bfiliks/streamlit-lda-dashboard works
Project type and purpose
• This is a single‐page interactive data‐analysis dashboard built with Streamlit (Python) for NLP on YouTube comments. It is not a traditional multi‐service microservices app but a full‐stack “one‐process” web app: the same Python process provides both UI and business logic.
File structure
• Top level:
• app.py – the Streamlit entry point; contains all UI and processing code
• requirements.txt – lists Python dependencies (Streamlit, Gensim, TextBlob, Plotly, sklearn, pandas, etc.)
• sample CSV data and .gitignore, LICENSE, README.md
• No separate frontend folder – Streamlit generates the UI
• No separate backend folder or database – all data lives in memory or is uploaded/downloaded as CSV
README for additional insights
• Explains pipeline steps: CSV upload → text preprocessing → topic modeling (LDA) → sentiment analysis (TextBlob) → t-SNE clustering → interactive Plotly visualizations → CSV export
• Technologies: Python 3.9+, Streamlit, Gensim, TextBlob, Plotly, scikit-learn (t-SNE)
a. Main components:
• User’s Browser / Streamlit UI
• Streamlit Web Server (Python runtime)
• Data Ingestion Module (CSV upload & sample loader)
• NLP Preprocessing Module (tokenization, stopword removal)
• NLP Core Modules:
– Topic Modeling (Gensim LDA)
– Sentiment Analysis (TextBlob)
– Dimensionality Reduction (t-SNE via scikit-learn)
• Visualization Engine (Plotly charts embedded in Streamlit)
• Data Export Module (filtered/comment CSV download)
• External Dependencies (Python libraries from requirements.txt)
b. Relationships & data flows:
1. User uploads CSV via Streamlit UI →
2. Data Ingestion reads into pandas DataFrame →
3. Data passes to Preprocessing → cleans and tokenizes →
4a. Feeds LDA for topic assignments → returns topic labels
4b. Feeds TextBlob for sentiment scores
4c. Feeds vectorized text into t-SNE for cluster coordinates
5. Visualization Engine receives processed data → renders interactive charts in the browser
6. User may trigger “Export” → Data Export Module writes out filtered DataFrame as new CSV → prompts browser download
c. Architectural patterns & principles:
• Pipeline pattern (sequential data transformations)
• Single‐page application hosted entirely in a Python process
• Stateless between requests (all state lives in memory per session)
• Modular code organization within app.py (could be refactored into sub‐modules)
d. Key technologies & libraries:
• Streamlit (UI framework)
• pandas (data manipulation)
• Gensim (LDA topic modeling)
• TextBlob (sentiment analysis)
• scikit-learn (t-SNE)
• Plotly (interactive charts)
Essential details
• How tensors or DataFrames move between modules