# gosom/google-maps-scraper architecture

> Architecture diagram of the GitHub repository gosom/google-maps-scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gosom/google-maps-scraper
- Repository: https://github.com/gosom/google-maps-scraper

## Overview

Google Maps Scraper extracts business leads, reviews, contact details, and related place data through CLI, web, REST, and SaaS interfaces. The principal workflow is job intake, runner selection, queued or direct scraping, Google Maps extraction, enrichment, and persistence for CSV, database, or API access. The SaaS edition adds authenticated administration, job monitoring, and cloud-worker provisioning. This map emphasizes sampled runtime code; several documented web/API routes remain intentionally lightly connected where their callers were not included.

Main entry points: [`runner/databaserunner/databaserunner.go`](https://github.com/gosom/google-maps-scraper/blob/main/runner/databaserunner/databaserunner.go) (Database Runner), [`cmd/gmapssaas/cmdworker/cmd_worker.go`](https://github.com/gosom/google-maps-scraper/blob/main/cmd/gmapssaas/cmdworker/cmd_worker.go) (Worker Command), [`admin/routes.go`](https://github.com/gosom/google-maps-scraper/blob/main/admin/routes.go) (Admin Routes), [`admin/handlers_workers.go`](https://github.com/gosom/google-maps-scraper/blob/main/admin/handlers_workers.go) (Worker Handlers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_intake["Job Intake"]
  node_scraper_cli["Scraper CLI<br/>[main.go]"]
  node_saas_cli["SaaS CLI<br/>[main.go]"]
  node_web_api["Web REST API<br/>[api.go]"]
  node_web_ui["Web UI<br/>[web.go]"]
  node_job_service["Job Service<br/>[service.go]"]
end

subgraph group_execution["Scraping Execution"]
  node_runner_factory["Runner Factory<br/>[main.go]"]
  node_database_runner["Database Runner<br/>[databaserunner.go]"]
  node_worker_command["Worker Command<br/>[cmd_worker.go]"]
  node_scraper_manager["Scraper Manager<br/>[scraper.go]"]
  node_job_queue["Job Queue<br/>[rqueue.go]"]
  node_maps_extractor["Maps Extractor<br/>[entry.go]"]
  node_review_extractor["Review Extractor<br/>[reviews.go]"]
end

subgraph group_results["Results Storage"]
  node_postgres_provider[("Postgres Provider<br/>[provider.go]")]
  node_result_writer["Result Writer<br/>[resultwriter.go]"]
  node_job_repository[("Job Repository<br/>[sqlite.go]")]
end

subgraph group_saas["SaaS Operations"]
  node_admin_routes["Admin Routes<br/>[routes.go]"]
  node_auth_handlers["Auth Handlers<br/>[handlers_auth.go]"]
  node_admin_store[("Admin Store<br/>[postgres.go]")]
  node_worker_handlers["Worker Handlers"]
end

subgraph group_infrastructure["Worker Infrastructure"]
  node_provision_jobs["Provision Jobs<br/>[worker_jobs.go]"]
  node_worker_provisioner["Worker Provisioner<br/>[worker.go]"]
  node_cloud_provider["Cloud Provider<br/>[digitalocean.go]"]
end

node_cli_user(("CLI User"))
node_web_user(("Web User"))
node_api_client(("API Client"))
node_admin_user(("Admin User"))
node_google_maps{{"Google Maps"}}
node_postgres_db[("Postgres Database")]
node_cloud_api{{"Cloud API"}}

node_cli_user -->|"invokes"| node_scraper_cli
node_scraper_cli -->|"selects runner"| node_runner_factory
node_runner_factory -->|"constructs"| node_database_runner
node_worker_command -->|"starts"| node_scraper_manager
node_worker_command -->|"starts"| node_job_queue
node_database_runner -->|"creates provider"| node_postgres_provider
node_database_runner -->|"creates writer"| node_result_writer
node_database_runner -->|"opens connection"| node_postgres_db
node_worker_command -->|"connects"| node_postgres_db
node_maps_extractor -->|"fetches places"| node_google_maps
node_review_extractor -->|"fetches reviews"| node_google_maps
node_worker_handlers -->|"creates provisioner"| node_worker_provisioner
node_worker_handlers -->|"stores worker state"| node_admin_store
node_provision_jobs -->|"provisions worker"| node_worker_provisioner
node_provision_jobs -->|"updates state"| node_postgres_db
node_worker_provisioner -.->|"calls provider"| node_cloud_api
node_cloud_provider -.->|"implements provider"| node_cloud_api
node_admin_routes -->|"dispatches auth"| node_auth_handlers
node_admin_routes -->|"dispatches workers"| node_worker_handlers
node_auth_handlers -->|"reads sessions"| node_admin_store
node_admin_store -->|"persists records"| node_postgres_db
node_web_user -->|"uses"| node_web_ui
node_api_client -->|"calls"| node_web_api
node_admin_user -->|"uses"| node_admin_routes

click node_scraper_cli "https://github.com/gosom/google-maps-scraper/blob/main/main.go"
click node_saas_cli "https://github.com/gosom/google-maps-scraper/blob/main/cmd/gmapssaas/main.go"
click node_web_api "https://github.com/gosom/google-maps-scraper/blob/main/api/api.go"
click node_web_ui "https://github.com/gosom/google-maps-scraper/blob/main/web/web.go"
click node_job_service "https://github.com/gosom/google-maps-scraper/blob/main/web/service.go"
click node_runner_factory "https://github.com/gosom/google-maps-scraper/blob/main/main.go"
click node_database_runner "https://github.com/gosom/google-maps-scraper/blob/main/runner/databaserunner/databaserunner.go"
click node_worker_command "https://github.com/gosom/google-maps-scraper/blob/main/cmd/gmapssaas/cmdworker/cmd_worker.go"
click node_scraper_manager "https://github.com/gosom/google-maps-scraper/blob/main/scraper/scraper.go"
click node_job_queue "https://github.com/gosom/google-maps-scraper/blob/main/rqueue/rqueue.go"
click node_maps_extractor "https://github.com/gosom/google-maps-scraper/blob/main/gmaps/entry.go"
click node_review_extractor "https://github.com/gosom/google-maps-scraper/blob/main/gmaps/reviews.go"
click node_postgres_provider "https://github.com/gosom/google-maps-scraper/blob/main/postgres/provider.go"
click node_result_writer "https://github.com/gosom/google-maps-scraper/blob/main/postgres/resultwriter.go"
click node_job_repository "https://github.com/gosom/google-maps-scraper/blob/main/web/sqlite/sqlite.go"
click node_admin_routes "https://github.com/gosom/google-maps-scraper/blob/main/admin/routes.go"
click node_auth_handlers "https://github.com/gosom/google-maps-scraper/blob/main/admin/handlers_auth.go"
click node_admin_store "https://github.com/gosom/google-maps-scraper/blob/main/admin/postgres/postgres.go"
click node_worker_handlers "https://github.com/gosom/google-maps-scraper/blob/main/admin/handlers_workers.go"
click node_provision_jobs "https://github.com/gosom/google-maps-scraper/blob/main/rqueue/worker_jobs.go"
click node_worker_provisioner "https://github.com/gosom/google-maps-scraper/blob/main/infra/worker.go"
click node_cloud_provider "https://github.com/gosom/google-maps-scraper/blob/main/infra/digitalocean/digitalocean.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scraper_cli,node_saas_cli,node_web_api,node_web_ui,node_job_service,node_cli_user,node_web_user,node_api_client,node_admin_user toneBlue
class node_runner_factory,node_database_runner,node_worker_command,node_scraper_manager,node_job_queue,node_maps_extractor,node_review_extractor,node_postgres_db toneAmber
class node_postgres_provider,node_result_writer,node_job_repository,node_cloud_api toneMint
class node_admin_routes,node_auth_handlers,node_admin_store,node_worker_handlers toneRose
class node_provision_jobs,node_worker_provisioner,node_cloud_provider,node_google_maps toneIndigo
```

## Components

### Job Intake

- [Scraper CLI](https://github.com/gosom/google-maps-scraper/blob/main/main.go): `main.go`
- [SaaS CLI](https://github.com/gosom/google-maps-scraper/blob/main/cmd/gmapssaas/main.go): `cmd/gmapssaas/main.go`
- [Web REST API](https://github.com/gosom/google-maps-scraper/blob/main/api/api.go): `api/api.go`
- [Web UI](https://github.com/gosom/google-maps-scraper/blob/main/web/web.go): `web/web.go`
- [Job Service](https://github.com/gosom/google-maps-scraper/blob/main/web/service.go): `web/service.go`

### Scraping Execution

- [Runner Factory](https://github.com/gosom/google-maps-scraper/blob/main/main.go): `main.go`
- [Database Runner](https://github.com/gosom/google-maps-scraper/blob/main/runner/databaserunner/databaserunner.go): `runner/databaserunner/databaserunner.go`
- [Worker Command](https://github.com/gosom/google-maps-scraper/blob/main/cmd/gmapssaas/cmdworker/cmd_worker.go): `cmd/gmapssaas/cmdworker/cmd_worker.go`
- [Scraper Manager](https://github.com/gosom/google-maps-scraper/blob/main/scraper/scraper.go): `scraper/scraper.go`
- [Job Queue](https://github.com/gosom/google-maps-scraper/blob/main/rqueue/rqueue.go): `rqueue/rqueue.go`
- [Maps Extractor](https://github.com/gosom/google-maps-scraper/blob/main/gmaps/entry.go): `gmaps/entry.go`
- [Review Extractor](https://github.com/gosom/google-maps-scraper/blob/main/gmaps/reviews.go): `gmaps/reviews.go`

### Results Storage

- [Postgres Provider](https://github.com/gosom/google-maps-scraper/blob/main/postgres/provider.go): `postgres/provider.go`
- [Result Writer](https://github.com/gosom/google-maps-scraper/blob/main/postgres/resultwriter.go): `postgres/resultwriter.go`
- [Job Repository](https://github.com/gosom/google-maps-scraper/blob/main/web/sqlite/sqlite.go): `web/sqlite/sqlite.go`

### SaaS Operations

- [Admin Routes](https://github.com/gosom/google-maps-scraper/blob/main/admin/routes.go): `admin/routes.go`
- [Auth Handlers](https://github.com/gosom/google-maps-scraper/blob/main/admin/handlers_auth.go): `admin/handlers_auth.go`
- [Admin Store](https://github.com/gosom/google-maps-scraper/blob/main/admin/postgres/postgres.go): `admin/postgres/postgres.go`
- [Worker Handlers](https://github.com/gosom/google-maps-scraper/blob/main/admin/handlers_workers.go): `admin/handlers_workers.go`

### Worker Infrastructure

- [Provision Jobs](https://github.com/gosom/google-maps-scraper/blob/main/rqueue/worker_jobs.go): `rqueue/worker_jobs.go`
- [Worker Provisioner](https://github.com/gosom/google-maps-scraper/blob/main/infra/worker.go): `infra/worker.go`
- [Cloud Provider](https://github.com/gosom/google-maps-scraper/blob/main/infra/digitalocean/digitalocean.go): `infra/digitalocean/digitalocean.go`

### Other components

- **CLI User**
- **Web User**
- **API Client**
- **Admin User**
- **Google Maps**
- **Postgres Database**
- **Cloud API**

## Connections

- CLI User → Scraper CLI: invokes
- Scraper CLI → Runner Factory: selects runner
- Runner Factory → Database Runner: constructs
- Worker Command → Scraper Manager: starts
- Worker Command → Job Queue: starts
- Database Runner → Postgres Provider: creates provider
- Database Runner → Result Writer: creates writer
- Database Runner → Postgres Database: opens connection
- Worker Command → Postgres Database: connects
- Maps Extractor → Google Maps: fetches places
- Review Extractor → Google Maps: fetches reviews
- Worker Handlers → Worker Provisioner: creates provisioner
- Worker Handlers → Admin Store: stores worker state
- Provision Jobs → Worker Provisioner: provisions worker
- Provision Jobs → Postgres Database: updates state
- Worker Provisioner → Cloud API: calls provider
- Cloud Provider → Cloud API: implements provider
- Admin Routes → Auth Handlers: dispatches auth
- Admin Routes → Worker Handlers: dispatches workers
- Auth Handlers → Admin Store: reads sessions
- Admin Store → Postgres Database: persists records
- Web User → Web UI: uses
- API Client → Web REST API: calls
- Admin User → Admin Routes: uses
