# eslamaboushashaa/scrape-facebook-by-playwright architecture

> Architecture diagram of the GitHub repository eslamaboushashaa/scrape-facebook-by-playwright, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eslamaboushashaa/scrape-facebook-by-playwright
- Repository: https://github.com/eslamaboushashaa/scrape-facebook-by-playwright

## Overview

##### Project type and purpose
- This is a lightweight full-stack Python application whose job is to scrape Facebook page metrics, analyze them, and present insights via a web interface.

##### File structure
- Top-level files:
- facebook_backend.py – contains the scraping and analysis logic
- web_app.py — implements the Streamlit UI that calls into the backend
- requirements.txt — lists Python dependencies (Playwright, BeautifulSoup, pandas, Streamlit, etc.)
- setup.sh — bootstraps the environment (installs system deps, Python packages)
- README.md — describes features and tech stack

##### README for additional insights
- Core features:
- Scrape post date, reactions, comments, shares from public and private pages via Playwright + BeautifulSoup
- Load data into pandas DataFrame, clean and analyze it
- Render graphs and tables in a Streamlit web app for competitor analysis
- Technical stack: Python, OOP, Playwright, BeautifulSoup, pandas, Streamlit

a. Main components
1) External Data Source: Facebook pages (public/private)
2) Scraper module (Playwright)
3) Parser module (BeautifulSoup)
4) Data processor (pandas analytics)
5) Web UI (Streamlit)
6) Environment/orchestration (setup.sh, Python runtime)

b. Relationships & data flow
- User opens the Streamlit UI (web_app.py)
→ UI triggers a “scrape and analyze” request
→ web_app.py calls functions in facebook_backend.py
→ Scraper module uses Playwright to fetch page HTML
→ Parser (BeautifulSoup) extracts metrics into raw data
→ Data processor (pandas) cleans, aggregates, and computes insights
→ Processed data and charts are returned to web_app.py
→ Streamlit renders tables and graphs for the user

c. Architectural patterns & design principles
- Layered architecture:
– Presentation layer (Streamlit front-end)
– Service layer (facebook_backend.py orchestrates scraping + analysis)
– Infrastructure layer (external Facebook pages, environment setup)
- Single-responsibility modules:
– Scraping vs. parsing vs. data analysis vs. UI rendering each live in their own functions or submodules
- Minimal external dependencies—everything is managed via requirements.txt

d. Key technologies & frameworks
- Playwright: headless browsing, logging in (if needed), HTML retrieval
- BeautifulSoup: HTML parsing and data extraction
- pandas: DataFrame creation, cleaning, analysis, pivoting
- Streamlit: rapidly build and serve interactive web dashboards
- setup.sh: reproducible setup of Python virtualenv and system prerequisites

##### Putting it all together
- At the top, show “User Web Browser” → Streamlit UI
- Right of service layer: chain of Scraper → Parser → Data Processor
- Bottom: “Hosting/Deployment” environment block (setup.sh, virtualenv, Python runtime)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Systems
    ExternalData[(Facebook Pages)]:::external
    Browser("User Web Browser"):::external

    %% Presentation Layer
    subgraph "Presentation Layer"
        UI["Streamlit UI"]:::ui
    end

    %% Service & Backend Pipeline
    subgraph "Service Layer & Backend Pipeline"
        Service["Service Layer\n(facebook_backend.py)"]:::backend
        Scraper["Playwright Scraper"]:::backend
        Parser["BeautifulSoup Parser"]:::backend
        Processor["pandas Data Processor"]:::backend
    end

    %% Deployment Environment
    subgraph "Deployment Environment"
        Setup["setup.sh"]:::env
        Reqs["requirements.txt"]:::env
        Doc["README.md"]:::env
    end

    %% Data Flow
    Browser -->|"opens UI"| UI
    UI -->|"scrape & analyze request"| Service
    ExternalData -->|"HTML responses"| Scraper
    Service -->|"fetch HTML"| Scraper
    Scraper -->|"raw HTML"| Parser
    Parser -->|"parsed data"| Processor
    Processor -->|"processed data & charts"| Service
    Service -->|"return charts & tables"| UI

    %% Environment Links
    Setup --> UI
    Setup --> Service
    Reqs --> UI
    Reqs --> Service

    %% Click Events
    click UI "https://github.com/eslamaboushashaa/scrape-facebook-by-playwright/blob/main/web_app.py"
    click Service "https://github.com/eslamaboushashaa/scrape-facebook-by-playwright/blob/main/facebook_backend.py"
    click Scraper "https://github.com/eslamaboushashaa/scrape-facebook-by-playwright/blob/main/facebook_backend.py"
    click Parser "https://github.com/eslamaboushashaa/scrape-facebook-by-playwright/blob/main/facebook_backend.py"
    click Processor "https://github.com/eslamaboushashaa/scrape-facebook-by-playwright/blob/main/facebook_backend.py"
    click Setup "https://github.com/eslamaboushashaa/scrape-facebook-by-playwright/blob/main/setup.sh"
    click Reqs "https://github.com/eslamaboushashaa/scrape-facebook-by-playwright/blob/main/requirements.txt"
    click Doc "https://github.com/eslamaboushashaa/scrape-facebook-by-playwright/blob/main/README.md"

    %% Styles
    classDef ui fill:#e0ffe0,stroke:#008000,color:#004000
    classDef backend fill:#e0f0ff,stroke:#0000ff,color:#000080
    classDef external fill:#ffe0b2,stroke:#ff8000,color:#804000
    classDef env fill:#f0f0f0,stroke:#888888,color:#444444
```
