aymaan-mulla/gdpr-compliant-multiplatform-scraper

How aymaan-mulla/gdpr-compliant-multiplatform-scraper works

This repository is a collection of local scripts for collecting social-media URLs, obtaining post and reel data through Bright Data, organizing results by collection, downloading linked media, and generating Word reports. The principal flow is URL categorization, Bright Data snapshot retrieval, collection organization, then media enrichment; report generation is a separate downstream utility. The README names a broader stack, but the sampled package and source show no Express, GraphQL, Neo4j, Prisma, or Redis runtime. Some listed scripts and data directories were not sampled, so their links are omitted.

Main entry points: organize.js Collection organizer · main.js Post snapshot client · main2.js Reel snapshot client · list4.js URL categorizer

Components 21

URL preparation

Collection importer scraper.js
URL categorizer list4.js
Collection URL files collections
Post URL files posts
Reel URL files reels

Snapshot collection

Bright Data
Post snapshot client main.js
Reel snapshot client main2.js
Collection snapshot client collections/main.js
Post scrape files fol2
Reel scrape files fol3
CSV exports collections

Media organization

Report generation

Connections 24

Collection importer → Collection URL files: writes JSON scraper.js
Collection URL files → URL categorizer: reads URLs list4.js
URL categorizer → Post URL files: writes posts list4.js
URL categorizer → Reel URL files: writes reels list4.js
Post snapshot client → Bright Data: triggers snapshots main.js
Post snapshot client → Bright Data: polls and downloads main.js
Reel snapshot client → Bright Data: triggers snapshots main2.js
Reel snapshot client → Bright Data: polls and downloads main2.js
Post snapshot client → Post scrape files: writes JSON main.js
Reel snapshot client → Reel scrape files: writes JSON main2.js
Collection snapshot client → Bright Data: triggers dataset collections/main.js
Collection snapshot client → CSV exports: writes CSV collections/main.js
Post URL files → Collection organizer: reads collection URLs organize.js
Reel URL files → Collection organizer: reads collection URLs organize.js
Post scrape files → Collection organizer: loads post data organize.js
Reel scrape files → Collection organizer: loads reel data organize.js
Collection organizer → Organized scrape data: writes grouped data organize.js
Organized scrape data → Media downloader: reads media links organ2.js
Media downloader → Bright Data: fetches media organ2.js
Media downloader → Downloaded media: writes files organ2.js
Python collection linker → Organized scrape data: writes collection copies doc2.py
Video frame extraction → Frame transfer: produces frames
Frame transfer → Word report generator: provides frames
Word report generator → Word reports: writes DOCX te.py

More JavaScript diagrams: rhymond/product-compare-react · sindresorhus/yocto-spinner · fossee/online_test · uptick/react-keyed-file-browser · earthcomfy/lets-chat

Drawn by GitDiagram on September 30, 2026 from the repository's file tree, README and source files. Read as Markdown ·

How aymaan-mulla/gdpr-compliant-multiplatform-scraper works

Main entry points: organize.js Collection organizer · main.js Post snapshot client · main2.js Reel snapshot client · list4.js URL categorizer

Connections 24

Collection importer → Collection URL files: writes JSON scraper.js
Collection URL files → URL categorizer: reads URLs list4.js
URL categorizer → Post URL files: writes posts list4.js
URL categorizer → Reel URL files: writes reels list4.js
Post snapshot client → Bright Data: triggers snapshots main.js
Post snapshot client → Bright Data: polls and downloads main.js
Reel snapshot client → Bright Data: triggers snapshots main2.js
Reel snapshot client → Bright Data: polls and downloads main2.js
Post snapshot client → Post scrape files: writes JSON main.js
Reel snapshot client → Reel scrape files: writes JSON main2.js
Collection snapshot client → Bright Data: triggers dataset collections/main.js
Collection snapshot client → CSV exports: writes CSV collections/main.js
Post URL files → Collection organizer: reads collection URLs organize.js
Reel URL files → Collection organizer: reads collection URLs organize.js
Post scrape files → Collection organizer: loads post data organize.js
Reel scrape files → Collection organizer: loads reel data organize.js
Collection organizer → Organized scrape data: writes grouped data organize.js
Organized scrape data → Media downloader: reads media links organ2.js
Media downloader → Bright Data: fetches media organ2.js
Media downloader → Downloaded media: writes files organ2.js
Python collection linker → Organized scrape data: writes collection copies doc2.py
Video frame extraction → Frame transfer: produces frames
Frame transfer → Word report generator: provides frames
Word report generator → Word reports: writes DOCX te.py

Drawn by GitDiagram on September 30, 2026 from the repository's file tree, README and source files. Read as Markdown ·