# bnbong/qshing_server architecture

> Architecture diagram of the GitHub repository bnbong/qshing_server, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bnbong/qshing_server
- Repository: https://github.com/bnbong/qshing_server

## Overview

##### Project type and purpose
- This is a backend phishing-detection microservice (“Qshing Server”) built in Python 3.12 using FastAPI.
- It exposes a POST API endpoint (`/phishing-detection/analyze`) that accepts a URL, crawls the page (via a local WebDriver), extracts text and images, and runs a multimodal CNN+BERT model to predict phishing.

##### File structure
- Top-level:
- Dockerfile, docker-compose.yml, build-spec.yaml (containerization & deployment)
- alembic.ini + migrations/ (database schema migrations)
- scripts/ (db init, lint, format, test, dependency fetch)
- src/qshing_server/: the application code
- Under `src/qshing_server`:
- api/: FastAPI routers & dependency definitions
- core/: config loader, custom exceptions
- dto/: Pydantic request/response schemas
- service/: business logic
– parser/: HTML loader via WebDriver
– model/: preprocessor, tokenizer, Q-BERT model, model manager (loads checkpoint `.pt`)
– phishing_analyzer.py: orchestrates parsing + model inference
- db/: db_manager.py (connects to Mongo or SQL), ORM models
- utils/: logging, enums
- main.py: FastAPI app instantiation, middleware, router inclusion

##### Readme insights
- Emphasizes Docker usage (local vs. container runtime differences when crawling).
- Notes AI model file must be placed in `service/model`.
- Mentions environment variables (SECRET_KEY, MODEL_NAME).

- External Client (curl/Postman/browser)
- API Layer: Uvicorn + FastAPI server
- Router: phishing_routers
- Dependencies (injection of config, DB session)
- Service Layer
- Parser subcomponent: HTML loader (uses local Chrome WebDriver)
- Analyzer: coordinates feature extraction + inference
- Model Layer
- Preprocessor → Tokenizer → Q-BERT (CNN+BERT) modules
- Model Manager (loads checkpoint file)
- Data Layer
- Database (MongoDB or SQL via Alembic migrations)
- db_manager & ORM models for storing request logs or stats
- Infrastructure
- Docker container(s)
- Docker Compose / CI workflows
- External WebDriver dependency

b. Relationships & interactions
1. Client → HTTP POST → FastAPI router
2. Router → inject config & DB session → hands off URL to PhishingAnalyzer
3. Analyzer → Parser loads HTML via WebDriver → returns raw data
4. Analyzer → Preprocessor transforms data → Tokenizer encodes text → Q-BERT infers → returns `result` + `confidence`
5. Analyzer → Optionally store request & result to DB via db_manager
6. FastAPI → Pydantic DTO constructs response → returns JSON to Client

c. Architectural patterns & design principles
- Layered Architecture: clear separation between API, service logic, model inference, and persistence.
- Dependency Injection via FastAPI `Depends` for config and DB sessions.
- Single-responsibility modules (parser, analyzer, model manager).
- Containerized deployment (Docker + Compose) for environment parity.

d. Key technologies/frameworks
- FastAPI + Pydantic for REST API
- Uvicorn as ASGI server
- PyTorch (or TensorFlow) for CNN+BERT inference
- Alembic for migrations
- Docker & Docker Compose for containerization
- Selenium/Chromedriver (implied) for HTML crawling
- CI via GitHub Actions

## Diagram (Mermaid)

```mermaid
flowchart TD
    Client["External Client"]:::external

    subgraph "Docker Infrastructure"
      direction TB
      DockerContainer[/"App Container"/]:::infra
      DBContainer[/"DB Container"/]:::infra
      SeleniumService[/"WebDriver Service"/]:::infra
      DockerContainer --> DBContainer
      DockerContainer --> SeleniumService
      Dockerfile["Dockerfile"]:::doc
      Compose["docker-compose.yml"]:::doc
      BuildSpec["build-spec.yaml"]:::doc
    end

    subgraph "Application (Docker Container)"
      direction TB
      Uvicorn["Uvicorn ASGI Server"]:::infra
      Main["main.py"]:::api
      Uvicorn --> Main

      subgraph "API Layer"
        direction TB
        Routers["phishing_routers.py"]:::api
        Deps["deps.py"]:::api
        Main --> Routers
        Main --> Deps
      end

      subgraph "Core"
        direction TB
        Config["config.py"]:::core
        Exceptions["exceptions.py"]:::core
      end

      subgraph "DTO"
        direction TB
        BaseDTO["base.py"]:::dto
        PhishSchema["phishing_schema.py"]:::dto
      end

      subgraph "Service Layer"
        direction TB
        Analyzer["phishing_analyzer.py"]:::service
        Parser["html_loader.py"]:::service
        Analyzer --> Parser
      end

      subgraph "Model Layer"
        direction TB
        Preprocessor["preprocessor.py"]:::model
        Tokenizer["tokenizer.py"]:::model
        QBERT["qbert.py"]:::model
        Manager["model_manager.py"]:::model
        Analyzer --> Preprocessor --> Tokenizer --> QBERT
        QBERT --> Manager
      end

      subgraph "Data Layer"
        direction TB
        DBManager["db_manager.py"]:::db
        Models["models.py"]:::db
        Analyzer --> DBManager --> Models
      end

      subgraph "Utils"
        direction TB
        Logging["logging.py"]:::utils
        Enums["enums.py"]:::utils
      end
    end

    Client -->|POST /phishing-detection/analyze| Uvicorn
    Routers -->|Depends on| Config
    Routers --> Deps
    Deps --> DBManager
    Analyzer -->|returns result| Routers

    click Main "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/main.py"
    click Routers "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/api/phishing_routers.py"
    click Deps "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/api/deps.py"
    click Config "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/core/config.py"
    click Exceptions "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/core/exceptions.py"
    click BaseDTO "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/dto/base.py"
    click PhishSchema "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/dto/phishing_schema.py"
    click Analyzer "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/service/phishing_analyzer.py"
    click Parser "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/service/parser/html_loader.py"
    click Preprocessor "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/service/model/preprocessor.py"
    click Tokenizer "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/service/model/tokenizer.py"
    click QBERT "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/service/model/qbert.py"
    click Manager "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/service/model/model_manager.py"
    click DBManager "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/db/db_manager.py"
    click Models "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/db/models.py"
    click AlembicIni "https://github.com/bnbong/qshing_server/blob/main/alembic.ini"
    click Migrations "https://github.com/bnbong/qshing_server/tree/main/migrations/"
    click Dockerfile "https://github.com/bnbong/qshing_server/tree/main/Dockerfile"
    click Compose "https://github.com/bnbong/qshing_server/blob/main/docker-compose.yml"
    click BuildSpec "https://github.com/bnbong/qshing_server/blob/main/build-spec.yaml"
    click InitDB "https://github.com/bnbong/qshing_server/blob/main/scripts/init_db.py"
    click Logging "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/utils/logging.py"
    click Enums "https://github.com/bnbong/qshing_server/blob/main/src/qshing_server/utils/enums.py"

    classDef external fill:#f9f,stroke:#333,stroke-width:1px
    classDef infra fill:#ccf,stroke:#333,stroke-width:1px
    classDef api fill:#bbf,stroke:#333,stroke-width:1px
    classDef core fill:#bfb,stroke:#333,stroke-width:1px
    classDef dto fill:#fbf,stroke:#333,stroke-width:1px
    classDef service fill:#feb,stroke:#333,stroke-width:1px
    classDef model fill:#fbb,stroke:#333,stroke-width:1px
    classDef db fill:#bdf,stroke:#333,stroke-width:1px
    classDef utils fill:#dfb,stroke:#333,stroke-width:1px
    classDef doc fill:#ffd,stroke:#333,stroke-width:1px
```
