# albertincx/formatbot1 architecture

> Architecture diagram of the GitHub repository albertincx/formatbot1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/albertincx/formatbot1
- Repository: https://github.com/albertincx/formatbot1

## Overview

##### Project type and purpose
- This is a Telegram bot service (“InstantViewBot”) that takes channel post links lacking Telegram Instant View (IV), fetches and processes their content (via parsing and compression utilities), and delivers IV pages back to users.
- It exposes an HTTP API (src/api) for retrieving stored documents (text/html/MD) and uses a message queue for decoupled, asynchronous processing.

##### High-level architecture components
a. Telegram Bot Layer
- Listens to user updates via the Telegram Bot API (src/index.js, src/service/rabbitmq.js).
- Parses commands and messages (src/service/commands, src/keyboards, src/messages).
- Enqueues tasks onto RabbitMQ.
b. Message Queue
- RabbitMQ: decouples incoming tasks from processing workers.
c. Worker/Service Layer
- Consumes tasks from RabbitMQ (src/service/jobMessage.js).
- Uses parsing utilities (src/api/utils) to fetch, sanitize, convert to IV, compress, support long content, generate PDFs, etc.
- Interacts with external processors: puppeteer (src/api/utils/puppet.js), Mercury API (src/api/utils/mercury.js), Telegra.ph (src/api/utils/makeTelegraph.js).
d. Data Layer
- MongoDB via Mongoose schemas (src/api/models) to store parsed pages and metadata (schema.js, schemaUpdatedAt.js).
- Configuration in src/config/mongoose.js, vars.js.
e. HTTP API Layer
- Express routes (src/api/routes) to serve stored documents and trigger format operations.
- Models and routes grouped under src/api.
f. Scheduler/Cron
- cron.js triggers periodic reprocessing or cleanup.
g. External Dependencies
- Telegram Bot API, RabbitMQ broker, MongoDB, Telegra.ph, cloud-based link shorteners.

##### Relationships and data flow
1. User → Telegram Bot API → Bot Service
2. Bot Service places message on RabbitMQ queue
3. Worker Service pulls message → invokes parsing utils → stores result in MongoDB and/or sends data to Telegra.ph
4. Worker emits completion event → Bot Service retrieves and sends IV back to user
5. External HTTP clients can call Express API endpoints to retrieve stored documents

##### Architectural patterns and design principles
- Asynchronous, event-driven architecture (publish/subscribe via RabbitMQ)
- Layered separation: API layer, Service/Worker layer, Data layer, Config layer
- Single codebase implementing both HTTP API and background workers
- Modular utilities for parsing, sanitizing, image handling

##### Technologies, frameworks, libraries
- Node.js + Express (src/api)
- Mongoose for MongoDB (src/api/models, src/config/mongoose.js)
- amqplib or custom RabbitMQ wrapper (src/service/rabbitmq.js)
- Puppeteer, Mercury parser, Telegra.ph API
- Cron scheduling via node-cron (cron.js)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Users and Clients
    User["User"]:::external
    Client["HTTP Clients"]:::external

    %% Telegram Bot Layer
    subgraph "Telegram Bot Layer"
        IBot["Index (src/index.js)"]:::service
        RMQProducer["RabbitMQ Wrapper\n(src/service/rabbitmq.js)"]:::service
        Commands["Commands Module\n(src/service/commands)"]:::service
        Keyboards["Keyboards\n(src/keyboards/keyboards.js)"]:::service
        Messages["Messages\n(src/messages/format.js)"]:::service
    end

    %% Message Broker
    MQ["RabbitMQ"]:::broker

    %% Scheduler
    Cron["Scheduler (src/cron.js)"]:::service

    %% Worker Service
    subgraph "Worker Service"
        JobMsg["Job Consumer\n(src/service/jobMessage.js)"]:::service
        ivMaker["ivMaker.js"]:::service
        puppet["puppet.js"]:::service
        mercury["mercury.js"]:::service
        telegraph["makeTelegraph.js"]:::service
        pdf["pdf.js"]:::service
        sanitize["sanitize.js"]:::service
        dom["dom.js"]:::service
        fixImages["fixImages.js"]:::service
        links["links.js"]:::service
        parseHelper["parseHelper.js"]:::service
    end

    %% Data Layer
    subgraph "Data Layer"
        Mongo["MongoDB"]:::database
        Schema["schema.js"]:::database
        SchemaUpd["schemaUpdatedAt.js"]:::database
        MongooseCfg["mongoose.js\n(src/config/mongoose.js)"]:::service
        Vars["vars.js\n(src/config/vars.js)"]:::service
    end

    %% HTTP API Layer
    subgraph "HTTP API Layer"
        BotRoute["botroute.js\n(src/api/routes/botroute.js)"]:::service
        FormatRoute["format.js\n(src/api/routes/format.js)"]:::service
        APIUtilsBot["bot.js\n(src/api/utils/bot.js)"]:::service
        APIUtilsDB["db.js\n(src/api/utils/db.js)"]:::service
        APIUtilsLogger["logger.js\n(src/api/utils/logger.js)"]:::service
        APIUtilsIndex["index.js\n(src/api/utils/index.js)"]:::service
    end

    %% Configuration
    subgraph "Configuration"
        BotCfg["bot.js\n(src/config/bot.js)"]:::service
        BotTest["botTest.js\n(src/config/botTest.js)"]:::service
        Buttons["buttons.js\n(src/config/buttons.js)"]:::service
        Consts["consts.js\n(src/config/consts.js)"]:::service
        MongooseCfg2["mongoose.js\n(src/config/mongoose.js)"]:::service
        Vars2["vars.js\n(src/config/vars.js)"]:::service
    end

    %% External Services
    subgraph "External Services"
        TelegramAPI["Telegram Bot API"]:::external
        TeleAPI["Telegra.ph API"]:::external
        MercuryAPI["Mercury API"]:::external
        Puppeteer["Puppeteer"]:::external
    end

    %% User Interaction Flow
    User -->|HTTPS| TelegramAPI
    TelegramAPI --> IBot
    IBot --> Commands
    IBot --> Keyboards
    IBot --> Messages
    IBot --> RMQProducer
    RMQProducer -->|AMQP| MQ
    Cron -->|Scheduled| MQ

    %% Worker Processing
    MQ -->|AMQP| JobMsg
    JobMsg --> ivMaker
    JobMsg --> puppet
    JobMsg --> mercury
    JobMsg --> telegraph
    JobMsg --> pdf
    JobMsg --> sanitize
    JobMsg --> dom
    JobMsg --> fixImages
    JobMsg --> links
    JobMsg --> parseHelper
    JobMsg --> Mongo
    JobMsg --> TeleAPI
    JobMsg --> MercuryAPI
    JobMsg --> Puppeteer
    JobMsg -->|AMQP completion| MQ
    MQ -->|AMQP| RMQProducer
    RMQProducer --> TelegramAPI
    TelegramAPI -->|HTTPS| User

    %% HTTP API Data Flow
    Client -->|HTTPS| BotRoute
    Client -->|HTTPS| FormatRoute
    BotRoute --> APIUtilsBot
    FormatRoute --> APIUtilsBot
    APIUtilsBot --> APIUtilsDB
    APIUtilsDB --> Mongo

    %% Click Events
    click IBot "https://github.com/albertincx/formatbot1/blob/master/src/index.js"
    click RMQProducer "https://github.com/albertincx/formatbot1/blob/master/src/service/rabbitmq.js"
    click Commands "https://github.com/albertincx/formatbot1/tree/master/src/service/commands/"
    click Keyboards "https://github.com/albertincx/formatbot1/blob/master/src/keyboards/keyboards.js"
    click Messages "https://github.com/albertincx/formatbot1/blob/master/src/messages/format.js"
    click Cron "https://github.com/albertincx/formatbot1/blob/master/src/cron.js"
    click JobMsg "https://github.com/albertincx/formatbot1/blob/master/src/service/jobMessage.js"
    click ivMaker "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/ivMaker.js"
    click puppet "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/puppet.js"
    click mercury "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/mercury.js"
    click telegraph "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/makeTelegraph.js"
    click pdf "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/pdf.js"
    click sanitize "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/sanitize.js"
    click dom "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/dom.js"
    click fixImages "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/fixImages.js"
    click links "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/links.js"
    click parseHelper "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/parseHelper.js"
    click Schema "https://github.com/albertincx/formatbot1/blob/master/src/api/models/schema.js"
    click SchemaUpd "https://github.com/albertincx/formatbot1/blob/master/src/api/models/schemaUpdatedAt.js"
    click MongooseCfg "https://github.com/albertincx/formatbot1/blob/master/src/config/mongoose.js"
    click Vars "https://github.com/albertincx/formatbot1/blob/master/src/config/vars.js"
    click BotRoute "https://github.com/albertincx/formatbot1/blob/master/src/api/routes/botroute.js"
    click FormatRoute "https://github.com/albertincx/formatbot1/blob/master/src/api/routes/format.js"
    click APIUtilsBot "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/bot.js"
    click APIUtilsDB "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/db.js"
    click APIUtilsLogger "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/logger.js"
    click APIUtilsIndex "https://github.com/albertincx/formatbot1/blob/master/src/api/utils/index.js"
    click BotCfg "https://github.com/albertincx/formatbot1/blob/master/src/config/bot.js"
    click BotTest "https://github.com/albertincx/formatbot1/blob/master/src/config/botTest.js"
    click Buttons "https://github.com/albertincx/formatbot1/blob/master/src/config/buttons.js"
    click Consts "https://github.com/albertincx/formatbot1/blob/master/src/config/consts.js"
    click MongooseCfg2 "https://github.com/albertincx/formatbot1/blob/master/src/config/mongoose.js"
    click Vars2 "https://github.com/albertincx/formatbot1/blob/master/src/config/vars.js"

    %% Styles
    classDef service fill:#c8e6c9,stroke:#2e7d32,color:#000
    classDef external fill:#e0e0e0,stroke:#757575,color:#000
    classDef broker fill:#ffe0b2,stroke:#ef6c00,color:#000
    classDef database fill:#bbdefb,stroke:#1976d2,color:#000
```
