How firaold/customer-experience_for_fintech_apps works
Project type and purpose
• This is a data-pipeline / ETL module for scraping user reviews from the Google Play Store and performing sequential preprocessing, sentiment analysis, and keyword extraction.
• Its primary goal is to generate four datasets in data/:
• raw_reviews.csv
• clean_reviews.csv
• sentiment_reviews.csv
• thematic_reviews.csv
Main components
a. External Data Source
– Google Play Store API (via google-play-scraper library)
b. Web Scraper Component
– src/web_scrapper.py
– Outputs: raw_reviews.csv
c. Preprocessor Component
– src/pre_processor.py
– Operations: drop duplicates, handle missing, normalize dates, remove stopwords
– Outputs: clean_reviews.csv
d. Sentiment Analysis Component
– src/sentiment_analysis.py
– Uses: pandas, numpy (and any sentiment model/library)
– Outputs: sentiment_reviews.csv
e. Keyword / Thematic Extraction Component
– src/keyword_extractor.py
– Outputs: thematic_reviews.csv
f. Data Storage
– Local file system under data/ folder (CSV files)
Relationships and data flow
• Web Scraper → raw_reviews.csv (data store)
• raw_reviews.csv → Preprocessor → clean_reviews.csv
• clean_reviews.csv → Sentiment Analysis → sentiment_reviews.csv
• clean_reviews.csv → Keyword Extraction → thematic_reviews.csv
Architectural patterns & design principles
• Linear ETL pipeline (extract → transform → load)
• Modular scripts, each with single responsibility
• File-based staging between steps (loose coupling)
• Reproducibility via requirements.txt for environment
Essential technologies & libraries
• Python 3.8+
• google-play-scraper (data extraction)
• pandas, numpy (data manipulation)
• Optional: nltk/spacy for stopwords & keyword extraction
• Local CSV files as storage