How ffmemes/ff-backend works
Project Type and Purpose
• This is a backend service for a Telegram-based meme‐sharing/broadcasting system (“Fast Food Memes”).
• It comprises a Telegram Bot API layer, a core application (business logic + scheduled flows), data stores (Postgres + Redis), plus external integrations (VK, Instagram parsers, object storage for memes).
File Structure Analysis
• Top‐level:
• Dockerfiles & docker‐compose (Dockerfile, docker-compose.yml etc.) for containerized deployment
• alembic/ for DB migrations
• flow_deployments/ for scheduled/background flows (broadcasts, crossposting, parsers, stats, rewards, storage)
• src/ holds the main Python code:
– tgbot/ Telegram bot entrypoint, router, handlers, senders, dependencies
– broadcasts/, crossposting/, parsers/, rewards/, stats/, storage/ modules for core services
– recommendations/ for ranking/blending candidate memes
– database.py, redis.py, models.py for persistence layers
– main.py as application bootstrap
• Tests under tests/, linters/configs (.flake8, pyproject.toml), and supporting scripts (scripts/, justfile).
README Insights
• Uses Docker + docker-compose for local/dev/prod
• Postgres + Alembic migrations
• Tests require DB migrations, run via pytest
• Justfile and scripts orchestrate builds, migrations, backups
Diagramming the System
a. Main Components (Containers):
1. Telegram Bot Service (Python app)
• Polling/Webhook client to Telegram API
• Handler layer → business‐service layer
2. Flow Runner (scheduled/background tasks)
• Modules in flow_deployments/ orchestrate flows: broadcasts, crossposting, parsers, stats, rewards, storage ETL
3. Database (PostgreSQL)
• Schema managed via Alembic
• Tables: memes, users, stats, flows, logs, etc.
4. Cache/Message Broker (Redis)
• Caching, rate‐limits, ephemeral queues
5. Object Storage (e.g., S3 or local volume)
• Raw meme files, watermarked images, OCR outputs
6. External Services
• Telegram API
• VK/Instagram scrape/parsers
• Payment gateways (via bot payments)
b. Relationships & Data Flow:
• Telegram API → Bot Service → Handler → Service → DB/Redis → Handler → Telegram API
• Scheduled Flow Runner queries DB/Redis → performs ETL/analysis/parsing → stores back into DB or Storage → triggers broadcast/crosspost via Bot Service or external API
• Recommendations service reads stats from DB, uses ranking/blending logic, caches results in Redis or returns to bot for on‐demand recommendation
c. Architectural Patterns & Principles:
• Layered architecture: API layer (bot handlers) → business logic (services) → data access (models + database/redis)
• Modular flows: grouping background tasks by domain (broadcasts, stats, parsers)
• Infrastructure as code: Docker + docker-compose + Justfile + scripts
• CI integration: GitHub Actions for linting/tests
d. Key Technologies & Frameworks:
• Python 3.x, AsyncIO
• Telegram Bot API (aiohttp or aiogram style)
• SQLAlchemy + Alembic (Postgres)
• Redis for caching/queues
• Docker, Docker Compose, Just
• CI with GitHub Actions, linters (ruff)
Essential Details
• Clearly label service modules (e.g., “Meme Recommendation Engine,” “Broadcast Flow”)
• Orchestration mechanism (cron scheduler or Prefect-like orchestrator if used)
• Migration tool (Alembic) as part of CI/CD pipeline block
• Docker networking: services on ffmemes_network