# facundoolano/google-play-scraper architecture

> Architecture diagram of the GitHub repository facundoolano/google-play-scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facundoolano/google-play-scraper
- Repository: https://github.com/facundoolano/google-play-scraper

## Overview

##### Project type and purpose
- This is a Node.js library (an open-source scraper module) designed to programmatically fetch and parse data from the Google Play Store. Consumers import it into their own Node.js applications to invoke high-level methods (app, list, search, reviews, etc.) without worrying about HTTP details or HTML scraping.

##### File structure
- Top-level files
- index.js / index.d.ts – the public entry point exporting all methods.
- package.json, build & lint configs (.eslintrc, .editorconfig).
- lib/ – contains one file per public method (app.js, list.js, search.js, developer.js, suggest.js, reviews.js, similar.js, permissions.js, datasafety.js, categories.js) plus shared constants.js.
- lib/utils/ – low-level helpers (appList, mappingHelpers, processPages, request, scriptData, throttle).
- test/ – unit tests mirroring each module under lib/.

##### README for additional insights
- Public API methods correspond one-to-one with modules under lib/.
- Internally, each method:
a) builds HTTP requests (lib/utils/request.js)
b) applies optional caching (memoizee) and throttling (lib/utils/throttle.js)
c) parses HTML/JSON via mappingHelpers & scriptData
d) optionally paginates with processPages

a. Main components
- Consumer Application (your Node.js code)
- google-play-scraper Public API (index.js)
- Method Controllers (lib/app.js, lib/list.js, etc.)
- Utility Layer
– HTTP Requester (lib/utils/request.js)
– Throttle Controller (lib/utils/throttle.js)
– Caching/Memoization (via memoizee)
– HTML/JSON Parsers (mappingHelpers, scriptData)
– Pagination Handler (processPages)
- External Service
– Google Play Store (HTTPS endpoints and web pages)
b. Relationships and interactions
1. Consumer calls a public method (e.g. gplay.app({…}))
2. index.js delegates to the corresponding controller (app.js)
3. Controller consults constants (endpoints, parameters)
4. Controller invokes Requester via Throttle Controller
5. Requester issues HTTP GET/POST to Google Play
6. Response piped through Parsers (mappingHelpers, scriptData)
7. If fullDetail or pagination is requested, Pagination Handler loops back to Requester
8. Parsed JS objects returned up the stack to Consumer
9. Caching layer intercepts repeated calls to shortcut steps 4–7
c. Architectural patterns & principles
- Modular separation: one file per feature (Single Responsibility)
- Layered design: Public API → Controllers → Utilities → External Service
- Cross-cutting concerns abstracted (throttling, caching)
- Test coverage mirroring modules for reliability
d. Key technologies & libraries
- Node.js (CommonJS / ES modules)
- HTTP requests via built-in fetch or request library
- memoizee for caching (memoized functions)
- In-house throttle implementation (lib/utils/throttle.js)
- Mocha/Jest (test/) for unit tests

##### Elements
- Consumer → Public API → Controller → Utils → Google Play → back up the stack.
- Blue for public interfaces (index.js)
- Green for feature controllers (app.js, list.js, etc.)
- Yellow for utilities (request, throttle, parsers)
- Orange for external systems (Google Play)
- Grey for Consumer code

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Consumer Layer
    Consumer["Consumer Application"]:::consumer

    %% Public API Layer
    PublicAPI["Public API\n(index.js)"]:::public

    %% Feature Controllers Layer
    subgraph "Feature Controllers"
        direction TB
        App["app.js"]:::controller
        List["list.js"]:::controller
        Search["search.js"]:::controller
        Developer["developer.js"]:::controller
        Suggest["suggest.js"]:::controller
        Reviews["reviews.js"]:::controller
        Similar["similar.js"]:::controller
        Permissions["permissions.js"]:::controller
        DataSafety["datasafety.js"]:::controller
        Categories["categories.js"]:::controller
    end

    %% Shared Constants
    Constants["constants.js"]:::util

    %% Utility Layer
    subgraph "Utility Layer"
        direction TB
        Request["request.js"]:::util
        Throttle["throttle.js"]:::util
        Cache["memoizee (caching)"]:::util
        Pagination["processPages.js"]:::util
        ScriptData["scriptData.js"]:::util
        Mapping["mappingHelpers.js"]:::util
        AppList["appList.js"]:::util
    end

    %% External Service
    External["Google Play Store"]:::external

    %% Connections
    Consumer -->|"calls API"| PublicAPI
    PublicAPI -->|"delegates to"| App
    PublicAPI --> App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories
    App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories -->|"reads constants"| Constants
    App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories -->|"requests via throttle"| Throttle
    Throttle -->|"controls"| Request
    Request -->|"HTTP Request"| External
    External -->|"response HTML/JSON"| Request
    Request -->|"raw response"| ScriptData
    Request -->|"raw response"| Mapping
    ScriptData -->|"parsed data"| App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories
    Mapping -->|"parsed data"| App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories
    App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories -->|"returns data"| PublicAPI
    PublicAPI -->|"returns result"| Consumer
    App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories -->|"memoized wrapper"| Cache
    Cache -.->|"cached result"| App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories
    App & List & Search & Developer & Suggest & Reviews & Similar & Permissions & DataSafety & Categories -->|"pagination needed"| Pagination
    Pagination -.->|"loops back"| Request

    %% Click Events
    click PublicAPI "https://github.com/facundoolano/google-play-scraper/blob/main/index.js"
    click App "https://github.com/facundoolano/google-play-scraper/blob/main/lib/app.js"
    click List "https://github.com/facundoolano/google-play-scraper/blob/main/lib/list.js"
    click Search "https://github.com/facundoolano/google-play-scraper/blob/main/lib/search.js"
    click Developer "https://github.com/facundoolano/google-play-scraper/blob/main/lib/developer.js"
    click Suggest "https://github.com/facundoolano/google-play-scraper/blob/main/lib/suggest.js"
    click Reviews "https://github.com/facundoolano/google-play-scraper/blob/main/lib/reviews.js"
    click Similar "https://github.com/facundoolano/google-play-scraper/blob/main/lib/similar.js"
    click Permissions "https://github.com/facundoolano/google-play-scraper/blob/main/lib/permissions.js"
    click DataSafety "https://github.com/facundoolano/google-play-scraper/blob/main/lib/datasafety.js"
    click Categories "https://github.com/facundoolano/google-play-scraper/blob/main/lib/categories.js"
    click Constants "https://github.com/facundoolano/google-play-scraper/blob/main/lib/constants.js"
    click Request "https://github.com/facundoolano/google-play-scraper/blob/main/lib/utils/request.js"
    click Throttle "https://github.com/facundoolano/google-play-scraper/blob/main/lib/utils/throttle.js"
    click Pagination "https://github.com/facundoolano/google-play-scraper/blob/main/lib/utils/processPages.js"
    click ScriptData "https://github.com/facundoolano/google-play-scraper/blob/main/lib/utils/scriptData.js"
    click Mapping "https://github.com/facundoolano/google-play-scraper/blob/main/lib/utils/mappingHelpers.js"
    click AppList "https://github.com/facundoolano/google-play-scraper/blob/main/lib/utils/appList.js"

    %% Styles
    classDef consumer fill:#dddddd,stroke:#333,stroke-width:1px
    classDef public fill:#bbdefb,stroke:#333,stroke-width:1px
    classDef controller fill:#c8e6c9,stroke:#333,stroke-width:1px
    classDef util fill:#fff59d,stroke:#333,stroke-width:1px
    classDef external fill:#ffccbc,stroke:#333,stroke-width:1px
```
