How alexmudrak/solscan-parser works
Project type and purpose
• This is a Python-based CLI application whose sole job is to scrape blockchain data from the Solscan website via Selenium, parse it, and then push it into a Google Sheet via the Google Sheets API.
File structure
• Top‐level scripts:
• src/main.py and src/main_speed.py – entry points
• src/core:
• config.py – loads environment variables and constants
• logger.py – sets up application logging
• src/schemas:
• parse_schemas.py – defines data models or validation schemas
• src/services:
• solscan_parser.py – contains the Selenium logic to fetch and parse Solscan pages
• google_sheets.py – handles authentication and CRUD operations against Google Sheets
• Configuration files:
• .env.example – shows required env vars (e.g., Google credentials, sheet ID)
• credentials.json (external) – OAuth2 client secrets for Google API
README for additional insights
• The application:
1. Uses Selenium WebDriver to navigate Solscan
2. Extracts transaction or block data
3. Authenticates to Google Sheets API
4. Writes rows to a target spreadsheet
• Dependencies: Python 3.11+, selenium, google-api-python-client, other standard libraries
Main components & interactions
a. Internal components:
• CLI Runner (main.py)
• Configuration Loader (core/config.py)
• Logger (core/logger.py)
• Solscan Scraper Service (services/solscan_parser.py)
• Data Schema Validator (schemas/parse_schemas.py)
• Google Sheets Service (services/google_sheets.py)
b. External systems:
• Solscan website (https://solscan.io)
• Google Sheets API endpoint
• Local file system (credentials.json, .env)
c. Data flow & relationships:
1. User invokes CLI Runner → it loads config & logger
2. Runner calls Solscan Scraper Service → launches Selenium WebDriver → fetches HTML pages from Solscan
3. Scraper outputs raw data → passes through Data Schema Validator → produces a typed data model
4. Validated data is forwarded to Google Sheets Service → authenticates using credentials.json → calls Google Sheets API → appends or updates sheet rows
d. Architectural patterns & principles:
• Single‐responsibility modules (each service does one thing)
• Layered structure: core (infrastructure), services (use-cases), schemas (domain validation)
• Dependency injection via configuration (env vars and credential files)