# johnbalvin/pyzill architecture

> Architecture diagram of the GitHub repository johnbalvin/pyzill, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/johnbalvin/pyzill
- Repository: https://github.com/johnbalvin/pyzill

## Overview

##### Project Type and Purpose
- This is an open-source Python library (“pyzill”) for scraping real-estate data from Zillow.
- It exposes functions to search listings (for_sale, for_rent, sold) and to fetch detailed info for homes or apartments by URL or ID, all via HTTP requests through rotating proxies.

##### File Structure Analysis
- Top-level:
- pyproject.toml – packaging/config
- README.md – usage & feature doc
- test.py – simple usage script
- src/pyzill package:
- parse.py – proxy URL builder & raw request logic
- search.py – search endpoints and parameter assembly
- details.py – detailed-page scraping logic (home vs. apartment)
- utils.py – shared helpers (e.g., JSON cleanup, rate-limiting)
- No database or frontend; it’s purely a library that issues HTTP calls and returns JSON-serializable Python dicts.

##### Key Architecture Insights from README
- Relies on rotating residential proxies (parse_proxy).
- Users integrate by importing pyzill and invoking functions; output is Python data or JSON files.

a. Main Components
- Client Application – user’s Python code or test.py script
- pyzill Library – four modules: parse, search, details, utils
- HTTP Request Layer – underlying HTTP client (requests or similar)
- Proxy Service – rotating residential proxy endpoint
- Zillow Web Service – Zillow’s public search and detail endpoints
- JSON Parser & Data Cleaner – transforms raw HTTP responses into Python dicts
- Local Storage (optional) – writing JSON files to disk
b. Relationships & Data Flow
1. Client calls pyzill.for_sale / for_rent / sold / get_from_home_url / get_from_home_id / get_from_department_*
2. Module search.py or details.py invokes parse.parse_proxy to construct proxy-aware HTTP client
3. HTTP Request Layer issues GET/POST via proxy to Zillow Web Service
4. Zillow returns JSON or HTML
5. Module parse.py or details.py parses/normalizes data, using utils.py helpers
6. Resulting Python dict is returned to client; optionally serialized to disk
c. Architectural Patterns & Principles
- Modular separation of concerns: parsing/proxy logic, search queries, detail scraping, and utilities clearly split
- Facade-style library API: user sees a single pyzill namespace with well-named functions
- Single responsibility per module for maintainability and extensibility
d. Key Technologies & Libraries
- Python 3.x
- HTTP client (requests or httpx)
- JSON serialization (built-in json module)
- Rotating proxy support (custom parse_proxy logic)
- Packaging via pyproject.toml / pip

## Diagram (Mermaid)

```mermaid
flowchart TB
    A["Client Application\n(test.py)"]:::client
    Config["pyproject.toml"]:::storage
    Doc["README.md"]:::storage
    PKG["Library Package Directory\n(src/pyzill/)"]:::library
    subgraph "pyzill Library Modules"
        direction TB
        B_parse["parse.py"]:::library
        B_search["search.py"]:::library
        B_details["details.py"]:::library
        B_utils["utils.py"]:::library
    end
    C["HTTP Request Layer"]:::library
    D["Proxy Service"]:::external
    E["Zillow Web Service"]:::external
    F["Local Storage\n(JSON file)"]:::storage

    A -->|calls for_sale/for_rent/sold| B_search
    A -->|calls get_from_home URL/ID| B_details
    B_search -->|uses parse_proxy| B_parse
    B_details -->|uses parse_proxy| B_parse
    B_parse -->|issues HTTP| C
    C -->|routes via proxy| D
    D -->|GET/POST request| E
    E -->|response JSON/HTML| C
    C -->|raw response| B_parse
    B_parse -->|normalize raw data| B_utils
    B_details -->|scrape + normalize| B_utils
    B_utils -->|return Python dict| A
    A -->|optional write JSON| F
    PKG -->|contains| B_parse
    PKG -->|contains| B_search
    PKG -->|contains| B_details
    PKG -->|contains| B_utils
    Config -.->|package config| PKG
    Doc -.->|documentation| PKG

    click A "https://github.com/johnbalvin/pyzill/blob/main/test.py"
    click PKG "https://github.com/johnbalvin/pyzill/tree/main/src/pyzill/"
    click B_parse "https://github.com/johnbalvin/pyzill/blob/main/src/pyzill/parse.py"
    click B_search "https://github.com/johnbalvin/pyzill/blob/main/src/pyzill/search.py"
    click B_details "https://github.com/johnbalvin/pyzill/blob/main/src/pyzill/details.py"
    click B_utils "https://github.com/johnbalvin/pyzill/blob/main/src/pyzill/utils.py"
    click Config "https://github.com/johnbalvin/pyzill/blob/main/pyproject.toml"
    click Doc "https://github.com/johnbalvin/pyzill/blob/main/README.md"

    classDef library fill:#e3f2fd,stroke:#90caf9
    classDef external fill:#ffebee,stroke:#ef9a9a
    classDef client fill:#e8f5e9,stroke:#a5d6a7
    classDef storage fill:#fff3e0,stroke:#ffcc80
```
