How joeyism/linkedin_scraper works
LinkedIn Scraper is an async Playwright library for collecting LinkedIn profiles, company pages and posts, and job listings. Its principal workflow is to establish or reuse an authenticated browser session, navigate LinkedIn pages, extract structured records, and return Pydantic models; job search first discovers job URLs. Scrapers share browser-page utilities and progress callbacks. The README also documents manual and credential login, cookie/session reuse, and progress reporting. Some scraper implementations are only partially excerpted, so edges to models and shared layers are limited to relationships visible in the excerpts or README.
Main entry points: base.py Base scraper · company_posts.py Company posts scraper · person.py Profile scraper · company.py Company scraper
Components 17
Browser and authentication
- Browser manager
linkedin_scraper/core/browser.py - Authentication
linkedin_scraper/core/auth.py
Scraping workflows
- Profile scraper
linkedin_scraper/scrapers/person.py - Company scraper
linkedin_scraper/scrapers/company.py - Company posts scraper
linkedin_scraper/scrapers/company_posts.py - Job search scraper
linkedin_scraper/scrapers/job_search.py - Job scraper
linkedin_scraper/scrapers/job.py
Structured data
- Person records
linkedin_scraper/models/person.py - Company records
linkedin_scraper/models/company.py - Post records
linkedin_scraper/models/post.py - Job records
linkedin_scraper/models/job.py
Shared runtime support
- Base scraper
linkedin_scraper/scrapers/base.py - Browser utilities
linkedin_scraper/core/utils.py - Scraper exceptions
linkedin_scraper/core/exceptions.py - Progress callbacks
linkedin_scraper/callbacks.py
Other components
- Library user
Connections 29
- Library user → Browser manager: opens session
README.md - Browser manager → Authentication: loads session
README.md - Authentication → LinkedIn: authenticates
linkedin_scraper/core/auth.py - Library user → Profile scraper: requests profile
README.md - Library user → Company scraper: requests company
README.md - Library user → Company posts scraper: requests posts
README.md - Library user → Job search scraper: searches jobs
README.md - Library user → Job scraper: requests job
README.md - Profile scraper → LinkedIn: navigates and extracts
- Company scraper → LinkedIn: navigates and extracts
- Company posts scraper → LinkedIn: navigates and extracts
linkedin_scraper/scrapers/company_posts.py - Job search scraper → LinkedIn: searches listings
linkedin_scraper/scrapers/job_search.py - Job scraper → LinkedIn: navigates and extracts
linkedin_scraper/scrapers/job.py - Profile scraper → Base scraper: uses helpers
linkedin_scraper/scrapers/person.py - Company scraper → Base scraper: uses helpers
linkedin_scraper/scrapers/company.py - Company posts scraper → Base scraper: uses helpers
linkedin_scraper/scrapers/company_posts.py - Job search scraper → Base scraper: uses helpers
linkedin_scraper/scrapers/job_search.py - Job scraper → Base scraper: uses helpers
linkedin_scraper/scrapers/job.py - Base scraper → Browser utilities: delegates helpers
linkedin_scraper/scrapers/base.py - Base scraper → Authentication: checks login
linkedin_scraper/scrapers/base.py - Base scraper → Scraper exceptions: raises errors
linkedin_scraper/scrapers/base.py - Authentication → Scraper exceptions: raises errors
linkedin_scraper/core/auth.py - Profile scraper → Person records: returns records
linkedin_scraper/scrapers/person.py - Company scraper → Company records: returns records
linkedin_scraper/scrapers/company.py - Company posts scraper → Post records: creates records
linkedin_scraper/scrapers/company_posts.py - Job scraper → Job records: creates records
linkedin_scraper/scrapers/job.py - Base scraper → Progress callbacks: reports progress
linkedin_scraper/scrapers/base.py - Job search scraper → Progress callbacks: reports progress
linkedin_scraper/scrapers/job_search.py - Company posts scraper → Progress callbacks: reports progress
linkedin_scraper/scrapers/company_posts.py