How gen-git-26/rreview_analyzer works
Main Components
• Data Ingestion & Preprocessing:
• “data” folder where scraped CSV files are stored (both raw data in “data/scraped_data” and cleaned data files).
• Data cleaning and aggregation process that leads to cleaned CSV files.
• Text Analysis using NLP/LLM Models:
• Two distinct modules: one for labeling sentiment using the “tabularisai/robust-sentiment-analysis” model and another module for sentiment training using the “distilbert-base-uncased” model.
• Image Analysis (Computer Vision):
• Two parallel image analysis tracks: one using CLIP and the other using batch classification with ResNet.
• Integration and Result Aggregation:
• User Interface / Chat-SQL Component:
• Interactive “app.py” (in the final folder) built with Streamlit, which acts as the front-end to query the analyzed data with SQL-style questions.
• Experiment & Development Artifacts:
• Optionally, include the “notebooks” folder to indicate iterative experimentation and development, showing that initial prototypes (app_v1.py, app_v2.py, etc.) and various experiment notebooks contributed to the refinement of components.
Relationships and Interactions
• From raw scraped data → Data Cleaning → Processed Data
• From Processed Data → Parallel branches for NLP analysis and Image analysis.
• From these analysis modules, converge outputs into a combined insights store.
• Dependencies between analysis sub-components and note that they share the same cleaned data resources.
Architectural Patterns & Design Principles
• Modularity: separate modules for data ingestion, text analysis, image analysis, and presentation.
• That the project uses a pipeline-style architecture where each stage (data collection, analysis, integration, UI) passes along its output to the next.
• That the project uses external services/models (LLM and CV frameworks) and leverages pre-trained models as part of its processing.
Technologies, Frameworks, and Libraries
• Mention Python (implied by usage of Jupyter Notebooks and .py files).
• Streamlit for building the interactive application.
• References to the pre-trained models: “tabularisai/robust-sentiment-analysis” and “distilbert-base-uncased” for NLP; CLIP and ResNet for image analysis.
• Optionally, include any data storage/SQL components (e.g., the “reviews.db” in the csv_to_sql folder) where converted CSV data is stored for rapid queries.