# alexmudrak/solscan-parser architecture

> Architecture diagram of the GitHub repository alexmudrak/solscan-parser, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alexmudrak/solscan-parser
- Repository: https://github.com/alexmudrak/solscan-parser

## Overview

##### Project type and purpose
- This is a Python-based CLI application whose sole job is to scrape blockchain data from the Solscan website via Selenium, parse it, and then push it into a Google Sheet via the Google Sheets API.

##### File structure
- Top‐level scripts:
- src/main.py and src/main_speed.py – entry points
- src/core:
- config.py – loads environment variables and constants
- logger.py – sets up application logging
- src/schemas:
- parse_schemas.py – defines data models or validation schemas
- src/services:
- solscan_parser.py – contains the Selenium logic to fetch and parse Solscan pages
- google_sheets.py – handles authentication and CRUD operations against Google Sheets
- Configuration files:
- .env.example – shows required env vars (e.g., Google credentials, sheet ID)
- credentials.json (external) – OAuth2 client secrets for Google API

##### README for additional insights
- The application:
1. Uses Selenium WebDriver to navigate Solscan
2. Extracts transaction or block data
3. Authenticates to Google Sheets API
4. Writes rows to a target spreadsheet
- Dependencies: Python 3.11+, selenium, google-api-python-client, other standard libraries

##### Main components & interactions
a. Internal components:
- CLI Runner (main.py)
- Configuration Loader (core/config.py)
- Logger (core/logger.py)
- Solscan Scraper Service (services/solscan_parser.py)
- Data Schema Validator (schemas/parse_schemas.py)
- Google Sheets Service (services/google_sheets.py)
b. External systems:
- Solscan website (https://solscan.io)
- Google Sheets API endpoint
- Local file system (credentials.json, .env)
c. Data flow & relationships:
1. User invokes CLI Runner → it loads config & logger
2. Runner calls Solscan Scraper Service → launches Selenium WebDriver → fetches HTML pages from Solscan
3. Scraper outputs raw data → passes through Data Schema Validator → produces a typed data model
4. Validated data is forwarded to Google Sheets Service → authenticates using credentials.json → calls Google Sheets API → appends or updates sheet rows
d. Architectural patterns & principles:
- Single‐responsibility modules (each service does one thing)
- Layered structure: core (infrastructure), services (use-cases), schemas (domain validation)
- Dependency injection via configuration (env vars and credential files)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Actor
    User["User"]:::external

    %% Internal Modules
    subgraph "Internal Modules"
        CLIRunner["CLI Runner"]:::internal
        ConfigLoader["Configuration Loader"]:::internal
        Logger["Logger"]:::internal
        subgraph "Scraper Layer"
            SolScraper["Solscan Scraper Service"]:::internal
            WebDriver["Selenium WebDriver"]:::selenium
        end
        Validator["Data Schema Validator"]:::internal
        SheetsService["Google Sheets Service"]:::internal
    end

    %% External Systems & Config
    SolscanSite["Solscan Website"]:::external
    GoogleAPI["Google Sheets API"]:::external
    ConfigStore[".env / credentials.json"]:::config

    %% Data Flow
    User -->|"invoke CLI"| CLIRunner
    CLIRunner -->|"load config"| ConfigLoader
    CLIRunner -->|"init logger"| Logger
    CLIRunner -->|"start scraping"| SolScraper
    SolScraper --> WebDriver
    WebDriver -->|"fetch HTML"| SolscanSite
    SolScraper -->|"raw data"| Validator
    Validator -->|"validated model"| SheetsService
    Validator -->|validationError| Logger
    SheetsService -->|"authenticate & append"| GoogleAPI
    SheetsService -->|"read credentials"| ConfigStore
    ConfigLoader -->|"read env vars"| ConfigStore

    %% Click Events
    click CLIRunner "https://github.com/alexmudrak/solscan-parser/blob/master/src/main.py"
    click CLIRunner "https://github.com/alexmudrak/solscan-parser/blob/master/src/main_speed.py"
    click ConfigLoader "https://github.com/alexmudrak/solscan-parser/blob/master/src/core/config.py"
    click Logger "https://github.com/alexmudrak/solscan-parser/blob/master/src/core/logger.py"
    click SolScraper "https://github.com/alexmudrak/solscan-parser/blob/master/src/services/solscan_parser.py"
    click Validator "https://github.com/alexmudrak/solscan-parser/blob/master/src/schemas/parse_schemas.py"
    click SheetsService "https://github.com/alexmudrak/solscan-parser/blob/master/src/services/google_sheets.py"
    click ConfigStore "https://github.com/alexmudrak/solscan-parser/blob/master/.env.example"

    %% Styles
    classDef internal fill:#D3E5FF,stroke:#0366d6,color:#000
    classDef external fill:#DFFFE0,stroke:#2a7f2a,color:#000
    classDef config fill:#FFF3D3,stroke:#D5A50E,color:#000
    classDef selenium fill:#E5FFE0,stroke:#75B13D,color:#000
    classDef error fill:#FFE5E5,stroke:#D63031,color:#000
```
