# ffmemes/ff-backend architecture

> Architecture diagram of the GitHub repository ffmemes/ff-backend, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ffmemes/ff-backend
- Repository: https://github.com/ffmemes/ff-backend

## Overview

##### Project Type and Purpose
- This is a backend service for a Telegram-based meme‐sharing/broadcasting system (“Fast Food Memes”).
- It comprises a Telegram Bot API layer, a core application (business logic + scheduled flows), data stores (Postgres + Redis), plus external integrations (VK, Instagram parsers, object storage for memes).

##### File Structure Analysis
- Top‐level:
- Dockerfiles & docker‐compose (`Dockerfile`, `docker-compose.yml` etc.) for containerized deployment
- `alembic/` for DB migrations
- `flow_deployments/` for scheduled/background flows (broadcasts, crossposting, parsers, stats, rewards, storage)
- `src/` holds the main Python code:
– `tgbot/` Telegram bot entrypoint, router, handlers, senders, dependencies
– `broadcasts/`, `crossposting/`, `parsers/`, `rewards/`, `stats/`, `storage/` modules for core services
– `recommendations/` for ranking/blending candidate memes
– `database.py`, `redis.py`, `models.py` for persistence layers
– `main.py` as application bootstrap
- Tests under `tests/`, linters/configs (`.flake8`, `pyproject.toml`), and supporting scripts (`scripts/`, `justfile`).

##### README Insights
- Uses Docker + docker-compose for local/dev/prod
- Postgres + Alembic migrations
- Tests require DB migrations, run via `pytest`
- Justfile and scripts orchestrate builds, migrations, backups

##### Diagramming the System
a. Main Components (Containers):
1. Telegram Bot Service (Python app)
- Polling/Webhook client to Telegram API
- Handler layer → business‐service layer
2. Flow Runner (scheduled/background tasks)
- Modules in `flow_deployments/` orchestrate flows: broadcasts, crossposting, parsers, stats, rewards, storage ETL
3. Database (PostgreSQL)
- Schema managed via Alembic
- Tables: memes, users, stats, flows, logs, etc.
4. Cache/Message Broker (Redis)
- Caching, rate‐limits, ephemeral queues
5. Object Storage (e.g., S3 or local volume)
- Raw meme files, watermarked images, OCR outputs
6. External Services
- Telegram API
- VK/Instagram scrape/parsers
- Payment gateways (via bot payments)

b. Relationships & Data Flow:
- Telegram API → Bot Service → Handler → Service → DB/Redis → Handler → Telegram API
- Scheduled Flow Runner queries DB/Redis → performs ETL/analysis/parsing → stores back into DB or Storage → triggers broadcast/crosspost via Bot Service or external API
- Recommendations service reads stats from DB, uses ranking/blending logic, caches results in Redis or returns to bot for on‐demand recommendation

c. Architectural Patterns & Principles:
- Layered architecture: API layer (bot handlers) → business logic (services) → data access (models + database/redis)
- Modular flows: grouping background tasks by domain (broadcasts, stats, parsers)
- Infrastructure as code: Docker + docker-compose + Justfile + scripts
- CI integration: GitHub Actions for linting/tests

d. Key Technologies & Frameworks:
- Python 3.x, AsyncIO
- Telegram Bot API (aiohttp or aiogram style)
- SQLAlchemy + Alembic (Postgres)
- Redis for caching/queues
- Docker, Docker Compose, Just
- CI with GitHub Actions, linters (ruff)

##### Essential Details
- Clearly label service modules (e.g., “Meme Recommendation Engine,” “Broadcast Flow”)
- Orchestration mechanism (cron scheduler or Prefect-like orchestrator if used)
- Migration tool (Alembic) as part of CI/CD pipeline block
- Docker networking: services on `ffmemes_network`

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Real-time Bot Service Layer
    subgraph "Real-time Bot Service" 
        direction TB
        BotService["Telegram Bot Service"]:::realtime
    end
    click BotService "https://github.com/ffmemes/ff-backend/tree/production/src/tgbot/"

    %% Background Flow Runner
    subgraph "Background Flow Runner" 
        direction TB
        FlowRunner["Flow Runner"]:::flows
        BroadcastFlow["Broadcast Flow"]:::flows
        CrosspostFlow["Crossposting Flow"]:::flows
        ParserFlow["Parsing Flow"]:::flows
        RewardsFlow["Rewards Flow"]:::flows
        StatsFlow["Statistics Flow"]:::flows
        StorageFlow["Storage ETL Flow"]:::flows
    end
    click FlowRunner "https://github.com/ffmemes/ff-backend/tree/production/flow_deployments/"
    click BroadcastFlow "https://github.com/ffmemes/ff-backend/blob/production/flow_deployments/broadcasts.py"
    click CrosspostFlow "https://github.com/ffmemes/ff-backend/blob/production/flow_deployments/crossposting.py"
    click ParserFlow "https://github.com/ffmemes/ff-backend/blob/production/flow_deployments/parsers.py"
    click RewardsFlow "https://github.com/ffmemes/ff-backend/blob/production/flow_deployments/rewards.py"
    click StatsFlow "https://github.com/ffmemes/ff-backend/blob/production/flow_deployments/stats.py"
    click StorageFlow "https://github.com/ffmemes/ff-backend/blob/production/flow_deployments/storage.py"

    %% Core Services under src/
    subgraph "Core Services" 
        direction TB
        BroadcastSvc["Broadcast Service"]:::flows
        CrosspostSvc["Crossposting Service"]:::flows
        ParsersCore["Parsers Modules"]:::flows
        RewardsSvc["Rewards Service"]:::flows
        StatsSvc["Statistics Service"]:::flows
        StorageCore["Storage Service"]:::flows
        RecEngine["Recommendations Engine"]:::realtime
    end
    click BroadcastSvc "https://github.com/ffmemes/ff-backend/blob/production/src/broadcasts/service.py"
    click CrosspostSvc "https://github.com/ffmemes/ff-backend/blob/production/src/crossposting/service.py"
    click ParsersCore "https://github.com/ffmemes/ff-backend/tree/production/src/flows/parsers/"
    click RewardsSvc "https://github.com/ffmemes/ff-backend/tree/production/src/flows/rewards/"
    click StatsSvc "https://github.com/ffmemes/ff-backend/tree/production/src/flows/stats/"
    click StorageCore "https://github.com/ffmemes/ff-backend/tree/production/src/storage/"
    click RecEngine "https://github.com/ffmemes/ff-backend/tree/production/src/recommendations/"

    %% Data Stores
    subgraph "Data Stores" 
        direction TB
        Postgres[(PostgreSQL)]:::datastore
        Redis[(Redis)]:::datastore
        ObjStorage[(Object Storage)]:::datastore
    end

    %% External Integrations
    subgraph "External Services" 
        direction TB
        TelegramAPI[("Telegram API")]:::external
        VK[("VK API")]:::external
        IG[("Instagram API")]:::external
        Payment[("Payment Gateway")]:::external
    end

    %% Persistence & Infra
    subgraph "Persistence & Infra" 
        direction TB
        Alembic["Alembic Migrations"]:::infra
        ORM["ORM Models"]:::infra
        RedisClient["Redis Integration"]:::infra
        AppBootstrap["App Bootstrap"]:::infra
        InfraCode["Docker & Compose"]:::infra
    end
    click Alembic "https://github.com/ffmemes/ff-backend/tree/production/alembic/"
    click ORM "https://github.com/ffmemes/ff-backend/blob/production/src/models.py"
    click RedisClient "https://github.com/ffmemes/ff-backend/blob/production/src/redis.py"
    click AppBootstrap "https://github.com/ffmemes/ff-backend/blob/production/src/main.py"
    click InfraCode "https://github.com/ffmemes/ff-backend/blob/production/docker-compose.yml"

    %% Connections Real-time Flow
    TelegramAPI -->|"HTTP API"| BotService
    BotService -->|"Router/Handlers"| RecEngine
    BotService -->|"SQLAlchemy"| Postgres
    BotService -->|"Redis Pub/Sub"| Redis
    BotService -->|"Send Updates"| TelegramAPI

    %% Recommendations
    RecEngine -->|"Reads Stats"| Postgres
    RecEngine -->|"Caches Candidates"| Redis
    RecEngine -->|"Returns Recommendations"| BotService

    %% Background Flow Runner Flows
    FlowRunner --> BroadcastFlow
    FlowRunner --> CrosspostFlow
    FlowRunner --> ParserFlow
    FlowRunner --> RewardsFlow
    FlowRunner --> StatsFlow
    FlowRunner --> StorageFlow

    BroadcastFlow --> BroadcastSvc
    CrosspostFlow --> CrosspostSvc
    ParserFlow --> ParsersCore
    RewardsFlow --> RewardsSvc
    StatsFlow --> StatsSvc
    StorageFlow --> StorageCore

    %% Flows to Data Stores & External
    BroadcastSvc -->|"SQL"| Postgres
    BroadcastSvc -->|"Redis"| Redis
    BroadcastSvc -->|"Bot Calls"| BotService

    CrosspostSvc -->|"Crosspost API"| VK
    CrosspostSvc -->|"Crosspost API"| IG
    CrosspostSvc -->|"SQL"| Postgres

    ParsersCore -->|"Fetches"| VK
    ParsersCore -->|"Fetches"| IG
    ParsersCore -->|"Fetches"| TelegramAPI
    ParsersCore -->|"Stores"| Postgres
    ParsersCore -->|"Queues"| Redis

    RewardsSvc -->|"Analyzes"| Postgres
    RewardsSvc -->|"Stores"| Postgres

    StatsSvc -->|"Aggregates"| Postgres
    StatsSvc -->|"Publishes"| Redis

    StorageCore -->|"Stores Files"| ObjStorage
    StorageCore -->|"Writes Metadata"| Postgres

    %% Infra and CI/CD
    Alembic -.->|"Manages Schema"| Postgres
    AppBootstrap --> BotService
    AppBootstrap --> RecEngine
    InfraCode -.-> BotService
    InfraCode -.-> FlowRunner

    %% Styles
    classDef realtime fill:#8ecae6,stroke:#023047,color:#023047;
    classDef flows fill:#a8dadc,stroke:#006d77,color:#006d77;
    classDef datastore fill:#ffb703,stroke:#fb8500,color:#fb8500;
    classDef external fill:#adb5bd,stroke:#6c757d,color:#495057;
    classDef infra fill:#d3c0cd,stroke:#8d86c9,color:#5d496c;
```
