How imdibr/adshield_tn works
Project type and purpose
• This is a data-driven ML pipeline (“Adshield”) for real-time scam-ad detection and user profiling on social media.
• It scrapes ads & user data from Facebook, Twitter, Instagram; stores & preprocesses it; trains ML models; serves live inference; and issues alerts.
File structure
• Top level:
• data/ (raw CSVs)
• model_training/ (training & evaluation scripts)
• src_userprofilling/ (all ETL, scraping, preprocessing, inference code)
• config.ini, requirements.txt (env & dependency setup)
• test_data.json (small sample)
• Scraper modules: twitter_scrapping.py, facebook_userprofiling.py, scrap_instgram_userdata.py, scrappy.py
• Pipeline orchestration: src_userprofilling/main.py, userprofiling.py
• Modeling: src_userprofilling/model.py, modelscamllm.py, scam_detection_text_bert.py
• Training: model_training/model_training.py, model_eval.py
Key architectural insights from README
• Data collection via APIs & scraping
• Storage in MongoDB/PostgreSQL
• ML techniques: Random Forest, Gradient Boosting, neural nets, BERT/RoBERTa
• Real-time alerting via WebSockets, Kafka/RabbitMQ