# nomado-nakama/ideas_manager architecture

> Architecture diagram of the GitHub repository nomado-nakama/ideas_manager, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nomado-nakama/ideas_manager
- Repository: https://github.com/nomado-nakama/ideas_manager

## Overview

##### Project type and purpose
- This is a backend-focused, microservice-style RAG (Retrieval-Augmented Generation) system packaged via Docker Compose.
- Its purpose is to let users send Instagram URLs to a Telegram bot, download and process that content (images, video, audio), store metadata in Postgres, and answer user queries by retrieving context via RAG.

##### File structure
Top-level directories & files:
- configs/ingestion_worker – Dockerfile & Python deps for the worker that processes new URLs
- configs/tgbot – Dockerfile & deps for the Telegram bot service
- docker-compose.yaml – orchestrates tgbot, ingestion_worker, Postgres
- migrations/ – SQL scripts to set up the database schema
- nn_ideas_manager/ – core Python package
– core/db/ – DB connection, config, helper functions
– core/tgbot/ – Telegram bot config and handlers (start, link)
– core/workers/process_undigested_urls/ – logic to consume undigested URLs, download content, write to DB
- main.py – entrypoint that picks service based on environment

##### README for additional insights
- High-level architecture calls out three main layers: Telegram Bot, Ingestion Worker, and Postgres + storage.
- External dependencies: Telegram API, Instagram endpoints for reels and posts, local file storage, subwhisperer for transcription.
- Future work notes around audio transcription & summarization.

##### Detail
- Migrations step during DB initialization.
- Volume mounts from containers to storage.
- Retry/queue logic in the ingestion worker.
- RAG retrieval path: “query → context builder → LLM embedding → DB lookup.”

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Actors and Services
    User["User"]:::external
    TelegramAPI["Telegram API"]:::external
    InstagramAPI["Instagram API"]:::external
    Whisper["Subwhisperer/Whisper"]:::external

    %% Services Group
    subgraph "Services"
        direction TB
        TGBot["Telegram Bot Service"]:::container
        Worker["Ingestion Worker Service"]:::container
        Postgres["PostgreSQL Database"]:::db
        Storage["File Storage (Volume)"]:::db
    end

    %% Entrypoint and Orchestration
    Entrypoint["Service Dispatcher\n(nn_ideas_manager/main.py)"]:::container
    Compose["Docker Compose\nOrchestration"]:::container
    Migrations["DB Migrations"]:::external

    %% Data Flows
    User -->|"/sendMessage"| TelegramAPI
    TelegramAPI -->|Webhook| TGBot
    TGBot -->|"INSERT undigested_urls"| Postgres
    Worker -->|"SELECT undigested_urls"| Postgres
    Worker -->|"Download media"| InstagramAPI
    Worker -->|"Store files"| Storage
    Worker -->|"Transcription"| Whisper
    Worker -->|"INSERT processed data"| Postgres
    TGBot -->|"RAG Query"| Postgres
    TGBot -->|"/sendMessage"| TelegramAPI
    Migrations -->|Apply migrations| Postgres
    Compose -->|Orchestrates| TGBot
    Compose -->|Orchestrates| Worker
    Compose -->|Orchestrates| Postgres
    Compose -->|Mounts volume| Storage
    Entrypoint -->|Starts service based on ENV| TGBot
    Entrypoint -->|Starts service based on ENV| Worker

    %% Click Events for component mappings
    click TGBot "https://github.com/nomado-nakama/ideas_manager/tree/main/configs/tgbot/Dockerfile"
    click TGBot "https://github.com/nomado-nakama/ideas_manager/blob/main/configs/tgbot/requirements.txt"
    click TGBot "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/tgbot/config.py"
    click TGBot "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/tgbot/handlers/start_handler.py"
    click TGBot "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/tgbot/handlers/link_handler.py"

    click Worker "https://github.com/nomado-nakama/ideas_manager/tree/main/configs/ingestion_worker/Dockerfile"
    click Worker "https://github.com/nomado-nakama/ideas_manager/blob/main/configs/ingestion_worker/requirements.txt"
    click Worker "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/workers/process_undigested_urls/main.py"
    click Worker "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/workers/process_undigested_urls/utils.py"

    click Postgres "https://github.com/nomado-nakama/ideas_manager/blob/main/docker-compose.yaml"
    click Postgres "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/db/config.py"
    click Postgres "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/db/connection.py"
    click Postgres "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/db/functions.py"
    click Migrations "https://github.com/nomado-nakama/ideas_manager/blob/main/migrations/001_init.sql"

    click Storage "https://github.com/nomado-nakama/ideas_manager/blob/main/docker-compose.yaml"
    click Storage "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/core/workers/process_undigested_urls/utils.py"

    click Entrypoint "https://github.com/nomado-nakama/ideas_manager/blob/main/nn_ideas_manager/main.py"
    click Compose "https://github.com/nomado-nakama/ideas_manager/blob/main/docker-compose.yaml"

    %% Styles
    classDef container fill:#B3D1FF,stroke:#333,stroke-width:1px
    classDef db fill:#C6F6D5,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef external fill:#E2E8F0,stroke:#333,stroke-width:1px,stroke-dasharray: 2 2
```
