# aykseldi/craiglistjobscrape architecture

> Architecture diagram of the GitHub repository aykseldi/craiglistjobscrape, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aykseldi/craiglistjobscrape
- Repository: https://github.com/aykseldi/craiglistjobscrape

## Overview

##### Project Type and Purpose
- Type: A Python-based automation script invoked by a system scheduler.
- Purpose: Periodically scrape a Craigslist jobs page, filter postings by keywords, build an HTML summary, and send it via Gmail API.

##### Main Components
a. Scheduler Layer
– Cron (and optional rtcwake)
– Responsible for triggering the script every hour.
b. Automation Script (craiglist.py)
– HTTP Client & HTML Parser
- Libraries: requests or urllib + BeautifulSoup
– Filter Logic
- Matches job titles against a configured keyword list
– HTML Report Generator
- Assembles filtered results into an email-ready HTML body
– Gmail API Client
- Google API credentials, OAuth2 flow, sendMessage endpoint
c. External Services
– Craigslist Website (HTTP endpoint)
– Google Gmail API (REST endpoint)
d. Recipients
– Target email address(es) receiving the report

##### Relationships and Data Flow
- Cron → (executes) → craiglist.py
- craiglist.py → (HTTP GET) → Craigslist
- Craigslist → (HTML) → craiglist.py
- craiglist.py → (parse/filter/generate) → HTML body
- craiglist.py → (HTTPS POST) → Gmail API
- Gmail API → (delivers email) → Recipient inbox
- (Optional) craiglist.py → (invokes) → rtcwake → (puts system to sleep)

##### Architectural Patterns and Design Principles
– Pipeline pattern: sequence of fetch → parse → filter → generate → send
– Loose coupling: the scheduler, scraping logic, and email dispatch are separate steps
– Reusability: the script could be extended to support multiple keyword sets or output formats

##### Technologies / Frameworks / Libraries
– Python 3 (core language)
– requests or urllib + BeautifulSoup (web scraping)
– google-api-python-client & oauth2client (Gmail integration)
– cron & rtcwake (scheduling & power management)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Scheduler Layer
    Cron["Cron<br/>(every hour)"]:::scheduler
    rtcwake["rtcwake<br/>(sleep 1h)"]:::scheduler
    Cron -->|"executes"| Script["craiglist.py"]:::script
    Cron -->|"invoke (optional)"| rtcwake

    %% Automation Script Container
    subgraph "craiglist.py Pipeline"
        direction TB
        Fetcher["Fetcher<br/>(HTTP Client)"]:::script
        Parser["Parser<br/>(HTML Parser)"]:::script
        Filter["Filter<br/>(Keyword Logic)"]:::script
        Generator["Generator<br/>(HTML Report)"]:::script
        Mailer["Mailer<br/>(Gmail API Client)"]:::script
    end

    %% Internal Flow
    Script -->|triggers| Fetcher
    Fetcher -->|HTTP GET| Craigslist
    Craigslist -->|HTML| Parser
    Parser -->|parsed data| Filter
    Filter -->|filtered jobs| Generator
    Generator -->|HTML body| Mailer
    Mailer -->|HTTPS POST| GmailAPI

    %% External Services
    Craigslist["Craigslist Website<br/>(HTTP Endpoint)"]:::external
    GmailAPI["Google Gmail API<br/>(REST Endpoint)"]:::external
    Recipient["Recipient Inbox"]:::recipient

    %% Email Delivery
    GmailAPI -->|delivers email| Recipient

    %% Documentation
    Documentation["Project Documentation<br/>(README.md)"]:::doc
    Documentation -.-> Cron

    %% Click Events
    click Script "https://github.com/aykseldi/craiglistjobscrape/blob/master/craiglist.py"
    click Documentation "https://github.com/aykseldi/craiglistjobscrape/blob/master/README.md"

    %% Styles
    classDef scheduler fill:#D0E8FF,stroke:#0366D6,color:#0366D6;
    classDef script fill:#E6FFEA,stroke:#28A745,color:#28A745;
    classDef external fill:#FFF4E5,stroke:#FF8C00,color:#FF8C00;
    classDef recipient fill:#F0F0F0,stroke:#6A6A6A,color:#6A6A6A;
    classDef doc fill:#F5E6FF,stroke:#6F42C1,color:#6F42C1;
```
