# overklassniy/stankin_antispam_bot architecture

> Architecture diagram of the GitHub repository overklassniy/stankin_antispam_bot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/overklassniy/stankin_antispam_bot
- Repository: https://github.com/overklassniy/stankin_antispam_bot

## Overview

##### Project Type and Purpose
- This is a full‑stack anti‑spam system combining machine‑learning pipelines, a Telegram bot interface, and a web‑based administrative panel.
- Purpose: collect and preprocess chat data; train and host spam‑detection models; monitor and moderate incoming messages in Telegram; provide a browser UI for stats, logs, and model management.

##### High‑Level Components
a. Data Layer
- Raw dataset files under data/
- Preprocessing notebooks (ipynb/) and utils/preprocessing.py
- Model artifacts in models/
b. ML Pipeline
- Jupyter notebooks orchestrate ETL, model training, fine‑tuning (RuBERT, LR, NB, RF)
- utils/predictions.py for inference calls
c. Bot Service
- bot.py, run_bot.py launch an Aiogram‑based async Telegram bot
- utils/apis.py and utils/basic.py manage Telegram API interactions
d. Web Panel Service
- panel/app.py, auth.py, main.py define a Flask app with Flask‑SQLAlchemy and Flask‑Login
- panel/templates, static assets (CSS/JS) for UI
- run_panel.py or waitress to deploy
e. Configuration & Orchestration
- .env + config.json for environment variables and runtime settings
- run.py for combined or custom startup flows
f. External Services
- Telegram Bot API
- OpenAI API (optional ChatGPT enrichment)
- Combot Anti‑Spam (CAS) and LOLS via feature flags

##### Component Relationships & Interactions
- The Bot Service subscribes to incoming Telegram messages, invokes prediction routines (utils/predictions.py → loaded models), and, based on thresholds, deletes or mutes users and notifies an admin chat.
- The Web Panel reads/writes to the same database (Flask‑SQLAlchemy models in panel/db_models.py), exposing stats (spam counts, muted users) and allowing admin operations (model threshold tuning, session management).
- The ML Pipeline is mostly offline: it pulls raw data from data/, writes trained model files to models/, which are then loaded by both Bot Service and Web Panel for inference or display.

##### Architectural Patterns and Principles
- Layered architecture:
1. Data Ingestion & Preprocessing layer
2. Model Training & Storage layer
3. Service layer: Bot and Web Panel as separate micro‑services
4. Persistence layer: SQLite (instance/db.sqlite) or fallback
- Clear separation of concerns: utils/ for shared code; ipynb/ for experiments; panel/ for web UI; bot.py for realtime handling.
- Configuration‑driven: all runtime toggles and ports are set in config.json/.env.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        DataStorage["Raw Data (data/)"]:::data
        PreprocessingNotebook["1. Подготовка данных.ipynb"]:::data
        PreprocessingUtils["preprocessing.py"]:::data
    end

    %% ML Pipeline
    subgraph "ML Pipeline"
        direction TB
        NotebookTrain["2. Обучение различных моделей.ipynb"]:::data
        NotebookFine1["3.1. Дообучение RuBert.ipynb"]:::data
        NotebookFine2["3.2. Дообучение RuBert с числовыми значениями.ipynb"]:::data
        NotebookEval["4. Дополнительные оценки.ipynb"]:::data
        InferenceUtils["predictions.py"]:::service
        ModelStorage["Model Artifacts (models/)"]:::data
    end

    %% Shared Utilities
    subgraph "Shared Utils"
        direction TB
        APIUtils["apis.py"]:::service
        BasicUtils["basic.py"]:::service
        ConfigWeb["config_web.py"]:::service
    end

    %% Services
    subgraph "Services"
        direction TB
        BotService["Telegram Bot Service"]:::service
        WebPanel["Flask Admin Panel"]:::frontend
    end

    %% Configuration & Orchestration
    subgraph "Configuration & Orchestration"
        direction TB
        RunAll["run.py"]:::service
        RunBot["run_bot.py"]:::service
        RunPanel["run_panel.py"]:::service
    end

    %% Persistence
    subgraph "Persistence Layer"
        direction TB
        Database["SQLite DB (instance/db.sqlite)"]:::db
    end

    %% External Services
    subgraph "External APIs"
        direction TB
        TelegramAPI["Telegram Bot API"]:::external
        OpenAIAPI["OpenAI API"]:::external
        CASLOLS["Combot Anti‑Spam / LOLS"]:::external
    end

    %% Metadata
    subgraph "Project Metadata"
        direction TB
        README["README.md"]:::metadata
        Requirements["requirements.txt"]:::metadata
        AsciiInfo[".info/ascii-art.txt"]:::metadata
    end

    %% Connections
    DataStorage -->|"provides raw data"| PreprocessingNotebook
    PreprocessingNotebook -->|"uses utils"| PreprocessingUtils
    PreprocessingNotebook -->|"outputs processed data"| NotebookTrain
    NotebookTrain --> NotebookFine1
    NotebookFine1 --> NotebookFine2
    NotebookFine2 --> NotebookEval
    NotebookEval -->|"saves models"| ModelStorage

    ModelStorage -->|"load models"| BotService
    ModelStorage -->|"load models"| WebPanel

    BotService -->|"logs spam events"| Database
    WebPanel -->|"reads/writes stats"| Database

    BotService -->|"webhook & actions"| TelegramAPI
    BotService -->|"optional enrichment"| OpenAIAPI
    BotService -->|"feature flags"| CASLOLS

    BotService -->|"calls inference"| InferenceUtils
    InferenceUtils --> ModelStorage

    BotService --> APIUtils
    BotService --> BasicUtils

    WebPanel --> ConfigWeb
    BotService --> ConfigWeb

    RunAll --> BotService
    RunAll --> WebPanel
    RunAll --> ConfigWeb

    RunBot --> BotService
    RunPanel --> WebPanel

    %% Click Events
    click DataStorage "https://github.com/overklassniy/stankin_antispam_bot/tree/master/data/"
    click PreprocessingNotebook "https://github.com/overklassniy/stankin_antispam_bot/blob/master/ipynb/1. Подготовка данных.ipynb"
    click PreprocessingUtils "https://github.com/overklassniy/stankin_antispam_bot/blob/master/utils/preprocessing.py"
    click NotebookTrain "https://github.com/overklassniy/stankin_antispam_bot/blob/master/ipynb/2. Обучение различных моделей.ipynb"
    click NotebookFine1 "https://github.com/overklassniy/stankin_antispam_bot/blob/master/ipynb/3.1. Дообучение RuBert.ipynb"
    click NotebookFine2 "https://github.com/overklassniy/stankin_antispam_bot/blob/master/ipynb/3.2. Дообучение RuBert с числовыми значениями.ipynb"
    click NotebookEval "https://github.com/overklassniy/stankin_antispam_bot/blob/master/ipynb/4. Дополнительные оценки.ipynb"
    click InferenceUtils "https://github.com/overklassniy/stankin_antispam_bot/blob/master/utils/predictions.py"
    click APIUtils "https://github.com/overklassniy/stankin_antispam_bot/blob/master/utils/apis.py"
    click BasicUtils "https://github.com/overklassniy/stankin_antispam_bot/blob/master/utils/basic.py"
    click ConfigWeb "https://github.com/overklassniy/stankin_antispam_bot/blob/master/utils/config_web.py"
    click BotService "https://github.com/overklassniy/stankin_antispam_bot/blob/master/bot.py"
    click RunBot "https://github.com/overklassniy/stankin_antispam_bot/blob/master/run_bot.py"
    click WebPanel "https://github.com/overklassniy/stankin_antispam_bot/blob/master/panel/app.py"
    click runPanelFuncs PanelAuth "panel/auth.py"
    click PanelAuth "https://github.com/overklassniy/stankin_antispam_bot/blob/master/panel/auth.py"
    click PanelMain "https://github.com/overklassniy/stankin_antispam_bot/blob/master/panel/main.py"
    click PanelModels "https://github.com/overklassniy/stankin_antispam_bot/blob/master/panel/db_models.py"
    click Templates "https://github.com/overklassniy/stankin_antispam_bot/tree/master/panel/templates"
    click StaticAssets "https://github.com/overklassniy/stankin_antispam_bot/tree/master/panel/static/"
    click RunAll "https://github.com/overklassniy/stankin_antispam_bot/blob/master/run.py"
    click RunPanel "https://github.com/overklassniy/stankin_antispam_bot/blob/master/run_panel.py"
    click README "https://github.com/overklassniy/stankin_antispam_bot/blob/master/README.md"
    click Requirements "https://github.com/overklassniy/stankin_antispam_bot/blob/master/requirements.txt"
    click AsciiInfo "https://github.com/overklassniy/stankin_antispam_bot/blob/master/.info/ascii-art.txt"

    %% Styles
    classDef data fill:#FFF2B8,stroke:#333,stroke-width:1px
    classDef service fill:#D0E8FF,stroke:#333,stroke-width:1px
    classDef frontend fill:#E8D0FF,stroke:#333,stroke-width:1px
    classDef db fill:#D2F8D2,stroke:#333,stroke-width:1px
    classDef external fill:#F0F0F0,stroke:#333,stroke-width:1px
    classDef metadata fill:#ECECEC,stroke:#333,stroke-width:1px
```
