How aatmanvaidya/sentiment-analysis-of-online-harassment-towards-women-wrestlers works
Data Collection Layer
• YouTube Scraper (from youtube_scrape directory)
• Twitter Scraper (from twitter_scrape directory)
• Instagram Scraper (from insta_scrape directory)
• Data flow from social media platforms to the scrapers
Data Processing & Storage Layer
• A "Data Transformation" component showing:
• CSV to Feather conversion
• Video ID extraction
• Comment processing
• A "Data Storage" component showing the intermediate data formats
Analysis Layer
• BERTweet
• Dolly
• Google MuRiL
• Hinglish Model
• LLaMA v2
• Twitter RoBERTa
• A "Text Analysis" component showing:
• Regex Pattern Matching
• Slur Detection (using slur_list.txt)
• HurtLex Integration (using hurtlex_HI.tsv)
Output Layer
• A "Results & Analysis" component showing:
• Sentiment Analysis Output
• Harassment Detection Results
• Visualization components for the analysis results
• Data collection at the top
• Processing in the middle
• Analysis and results at the bottom
Additional Elements
• Input/output file formats (CSV, TSV, Feather, etc.)
• Integration points with external services
• Model dependencies and interactions
Key Considerations:
How different notebooks interact with each component
Remember to include version information for key components and any external dependencies that are crucial to the system's operation. This will help in understanding the system's requirements and maintenance needs.