# hum-ae-n/claudecode architecture

> Architecture diagram of the GitHub repository hum-ae-n/claudecode, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hum-ae-n/claudecode
- Repository: https://github.com/hum-ae-n/claudecode

## Overview

##### Project Type and Purpose
- This is a full-stack Python application combining:
- A modular web-scraper engine (CLI-driven and programmatic API)
- A Flask-based web interface (dashboard, product listing, export)
- A local SQLite database for storage
- Purpose: scrape e-commerce product data with rate limiting, retry/circuit breaker logic, expose a CLI and real-time web dashboard, and allow data export.

##### File Structure
- Top-level
- run_web_app.py: entry point for Flask UI
- example_usage.py, test_demo.py: sample usage and tests
- requirements.txt, setup.py: dependencies and packaging
- scraper/ (core package)
- cli.py: command-line interface commands
- config.py: configuration loader (env vars & JSON)
- database.py: SQLite manager (connect, save, query, export)
- models.py: Product data model
- scraper.py: scraping engine (URL extraction, product scraping)
- utils.py: rate limiter, retry logic, circuit breaker
- web_app.py: Flask app instantiation, route definitions
- templates/: Jinja2 HTML views (dashboard, product list, detail, scrape form)
- Data files and sample databases at top level for demo.

##### README for Additional Insights
- Core components map to modules in scraper/
- Architecture section in README confirms modular design and highlights:
- Token bucket rate limiter
- Exponential backoff retry with jitter
- Circuit breaker states
- SQLite storage with indexing
- Flask UI routes and templates

##### Detail Without Overthinking
- Every module as a separate component inside appropriate cluster

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Presentation Tier
    subgraph "Presentation Tier"
        direction TB
        CLIUser["CLI User<br/>(Terminal)"]:::external
        Browser["Web User<br/>(Browser)"]:::external
        CLIInterface["CLI Interface<br/>(scraper/cli.py)"]:::ui
        WebEntry["Web App Entry<br/>(run_web_app.py)"]:::ui
        WebInterface["Flask App<br/>(scraper/web_app.py)"]:::ui
        Templates["Jinja2 Templates<br/>(scraper/templates/*.html)"]:::ui
        ExampleUsage["Example Usage<br/>(example_usage.py)"]:::ui
        TestDemo["Test Demo<br/>(test_demo.py)"]:::ui
    end

    %% Core Engine
    subgraph "Core Engine"
        direction TB
        ConfigLoader["Config Loader<br/>(scraper/config.py)"]:::core
        ScraperModule["Scraper Module<br/>(scraper/scraper.py)"]:::core
        Utils["Utilities<br/>(scraper/utils.py)"]:::core
    end

    %% Data Tier
    subgraph "Data Tier"
        direction TB
        DBManager["Database Manager<br/>(scraper/database.py)"]:::data
        ProductModel["Product Model<br/>(scraper/models.py)"]:::data
        SQLiteDB["SQLite Database File"]:::data
    end

    %% External Systems
    subgraph "External Systems"
        direction TB
        Ecommerce["E-commerce Websites"]:::external
    end

    %% Connections
    CLIUser -->|"uses"| CLIInterface
    CLIInterface -->|"invoke"| ScraperModule
    CLIInterface -->|"export/read/write"| DBManager
    ExampleUsage -->|"demo CLI"| CLIInterface
    TestDemo -->|"test CLI"| CLIInterface

    Browser -->|"HTTP request"| WebEntry
    WebEntry -->|"start Flask"| WebInterface
    WebInterface -->|"invoke"| ScraperModule
    WebInterface -->|"read/write"| DBManager
    WebInterface -->|"render"| Templates

    ScraperModule -->|"use config"| ConfigLoader
    ScraperModule -->|"use utilities"| Utils
    ScraperModule -->|"HTTP request"| Ecommerce
    ScraperModule -->|"save/read"| DBManager

    DBManager -->|"uses model"| ProductModel
    DBManager -->|"store data"| SQLiteDB

    %% Click Events
    click CLIInterface "https://github.com/hum-ae-n/claudecode/blob/main/scraper/cli.py"
    click WebInterface "https://github.com/hum-ae-n/claudecode/blob/main/scraper/web_app.py"
    click ScraperModule "https://github.com/hum-ae-n/claudecode/blob/main/scraper/scraper.py"
    click Utils "https://github.com/hum-ae-n/claudecode/blob/main/scraper/utils.py"
    click ConfigLoader "https://github.com/hum-ae-n/claudecode/blob/main/scraper/config.py"
    click DBManager "https://github.com/hum-ae-n/claudecode/blob/main/scraper/database.py"
    click ProductModel "https://github.com/hum-ae-n/claudecode/blob/main/scraper/models.py"
    click Templates "https://github.com/hum-ae-n/claudecode/tree/main/scraper/templates"
    click WebEntry "https://github.com/hum-ae-n/claudecode/blob/main/run_web_app.py"
    click ExampleUsage "https://github.com/hum-ae-n/claudecode/blob/main/example_usage.py"
    click TestDemo "https://github.com/hum-ae-n/claudecode/blob/main/test_demo.py"

    %% Styles
    classDef ui fill:#D6EAFE,stroke:#0288D1,stroke-width:2px;
    classDef core fill:#E8F5E9,stroke:#388E3C,stroke-width:2px;
    classDef data fill:#FFF3E0,stroke:#F57C00,stroke-width:2px;
    classDef external fill:#F5F5F5,stroke:#9E9E9E,stroke-width:2px;
```
