# phamphamh/linkedin_scraper architecture

> Architecture diagram of the GitHub repository phamphamh/linkedin_scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/phamphamh/linkedin_scraper
- Repository: https://github.com/phamphamh/linkedin_scraper

## Overview

##### Main Components
- Library Core (linkedin_scraper module): Represent each of the key modules as individual components. This includes:
- Person – for scraping individual user profiles.
- Company – for scraping company profiles.
- Job and JobSearch – to manage job listings and searches.
- Actions – which handles login functionality and possibly other shared routines.
- Support Modules – such as constants, selectors, jobs, objects.
- Selenium Driver Integration: Show that the web scraping relies on Selenium’s Chrome driver. This module is integrated as a dependency and is passed to each API object.
- API/Interface: Represent the exposed API that users interact with (as seen in the README samples). This includes the user’s scripts (from samples and test directories) that interact with the library.
- Configuration/Setup: Include elements like environment variable configuration (e.g., setting CHROMEDRIVER) and installation scripts (setup.py, requirements.txt).

##### Relationships and Interactions
- The user or sample script calls the API (creating Person, Company, etc.) by passing the URL and a Selenium driver.
- The Actions module provides helper functions (like login) that act on the Selenium driver.
- Each API object (Person, Company, etc.) contains its own scrape() method that interacts with the driver to extract data.
- Support modules, like selectors and constants, facilitate the scraping logic by abstracting page element locators and configuration details.
- Clearly denote data flow: user scripts → actions (login) and API classes → driver interactions → scraped data output.

##### Architectural Patterns and Design Principles
- Facade Pattern: The library provides a simplified and cohesive API that internally manages complex Selenium interactions.
- Separation of Concerns: The project separates responsibilities into distinct objects and helper modules (for scraping actions, constants, selectors, etc.), making the architecture modular.
- Dependency Injection: The Selenium driver is passed into API objects allowing for flexibility and testability.

##### Relevant Technologies
- Python as the programming language.
- Selenium WebDriver as the technology for browser automation (Chrome driver specifically).
- Standard packaging tools (setup.py, setup.cfg) and distribution via pip.
- Testing modules in the “test” directory and sample usage scripts in “samples” which illustrate how the API is consumed.

##### Putting It All Together
- Inside, create separate blocks for the API objects, group utility modules, and Actions.

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Global Nodes and Subgraphs

    subgraph "Configuration/Setup"
        direction TB
        setup("Setup Scripts"):::config
        dependency("Dependency Management"):::config
    end

    subgraph "API Interface"
        direction TB
        scripts("User Scripts (Samples & Tests)"):::client
    end

    subgraph "Linkedin Scraper Library"
        direction TB
        core("Core Module"):::core
        %% API Objects
        person("Person Scraper API"):::api
        company("Company Scraper API"):::api
        jobSearch("Job Search API"):::api
        job("Job API"):::api
        %% Actions and Selenium Integration
        subgraph "Actions & Selenium Integration"
            direction TB
            actions("Actions Module"):::integration
            selenium("Selenium Driver (Chrome)"):::integration
        end
        %% Support Modules
        subgraph "Support Modules"
            direction TB
            constants("Constants"):::utility
            selectors("Selectors"):::utility
            objects("Objects"):::utility
        end
    end

    %% Relationships

    %% Core to API Objects 
    core -->|"contains"| person
    core -->|"contains"| company
    core -->|"contains"| jobSearch
    core -->|"contains"| job

    %% User Scripts invoking API Objects
    scripts -->|"invokes"| person
    scripts -->|"invokes"| company
    scripts -->|"invokes"| jobSearch
    scripts -->|"invokes"| job

    %% API Objects using Actions Module for Selenium operations
    person -->|"calls"| actions
    company -->|"calls"| actions
    jobSearch -->|"calls"| actions
    job -->|"calls"| actions

    %% Actions Module managing Selenium Driver
    actions -->|"manages"| selenium

    %% API Objects utilizing Support Modules
    person -->|"utilizes"| constants
    company -->|"utilizes"| selectors
    jobSearch -->|"utilizes"| objects

    %% Configuration impacting Selenium Driver and Dependencies
    setup -->|"configures"| selenium
    scripts -->|"reads"| dependency

    %% Click Events

    click core "https://github.com/phamphamh/linkedin_scraper/tree/master/linkedin_scraper/"
    click person "https://github.com/phamphamh/linkedin_scraper/blob/master/linkedin_scraper/person.py"
    click company "https://github.com/phamphamh/linkedin_scraper/blob/master/linkedin_scraper/company.py"
    click jobSearch "https://github.com/phamphamh/linkedin_scraper/blob/master/linkedin_scraper/job_search.py"
    click job "https://github.com/phamphamh/linkedin_scraper/blob/master/linkedin_scraper/jobs.py"
    click actions "https://github.com/phamphamh/linkedin_scraper/blob/master/linkedin_scraper/actions.py"
    click constants "https://github.com/phamphamh/linkedin_scraper/blob/master/linkedin_scraper/constants.py"
    click selectors "https://github.com/phamphamh/linkedin_scraper/blob/master/linkedin_scraper/selectors.py"
    click objects "https://github.com/phamphamh/linkedin_scraper/blob/master/linkedin_scraper/objects.py"
    click scripts "https://github.com/phamphamh/linkedin_scraper/tree/master/samples/"
    click scripts "https://github.com/phamphamh/linkedin_scraper/tree/master/test/"
    click setup "https://github.com/phamphamh/linkedin_scraper/blob/master/setup.py"
    click setup "https://github.com/phamphamh/linkedin_scraper/blob/master/setup.cfg"
    click dependency "https://github.com/phamphamh/linkedin_scraper/blob/master/requirements.txt"

    %% Styles
    classDef core fill:#DCEEFB,stroke:#1E81B0,stroke-width:2px;
    classDef api fill:#B5FFFC,stroke:#1E81B0,stroke-width:2px;
    classDef integration fill:#FFE5B4,stroke:#E07A5F,stroke-width:2px;
    classDef utility fill:#F7D6E0,stroke:#C44536,stroke-width:2px;
    classDef client fill:#D0F0C0,stroke:#4CAF50,stroke-width:2px;
    classDef config fill:#E0BBE4,stroke:#957DAD,stroke-width:2px;
```
