# khanxmetu/interpol-red-dl architecture

> Architecture diagram of the GitHub repository khanxmetu/interpol-red-dl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/khanxmetu/interpol-red-dl
- Repository: https://github.com/khanxmetu/interpol-red-dl

## Overview

##### Main Components
- api-poller:
– Purpose: Polls the Interpol Red Notices API at regular intervals.
– Key functions: Establishes HTTP sessions (using customized headers from config-data), applies configurable query strategies (e.g., simple default search and brute-force search), and sends retrieved notice data to a messaging queue via RabbitMQ (see api_poller/api_request_session.py and query_strategy.py).
– Technology: Python, Dockerized via its own Dockerfile.

- notice-saver:
– Purpose: Consumes notices from RabbitMQ, downloads associated images, and performs additional processing (“offense type classification”) before saving the notice details into a database.
– Key functions: Consuming messages (using rabbitmq_client.py), classification (via BasicOffenseTypeClassifier and placeholder GPTOffenseTypeClassifier), and repository management for persistence.
– Technology: Python, with Docker and configuration files for header spoofing.

- notice-dashboard:
– Purpose: Provides an interactive visualization and data analytics interface (built with Streamlit) for viewing the collected notices and associated statistics.
– Key functions: UI Components for notice details, lists, maps, and charts (files in ui_components and pages directories), and database access through repositories.
– Technology: Python/Streamlit, Dockerized.

- Supporting components (External/Infrastructure):
– RabbitMQ: Acts as the message broker between api-poller and notice-saver.
– Database: Although not explicitly shown in the file tree with a dedicated directory, notice-saver and notice-dashboard reference “db.py” and repository modules that imply a common datastore for persisted notices.

##### Relationships and Data Flow
- The api-poller periodically retrieves data from the Interpol API and pushes notice information to a RabbitMQ queue.
- The notice-saver reads from the RabbitMQ queue, processes the data (including image downloading and offense classification), then saves it to the database.
- The notice-dashboard connects to the same database to display up-to-date notices and related visual analytics.
- Optional flows include configuration settings (like header spoofing) and logging/exception handling between components.

##### Architectural Patterns and Design Principles
- Separation of Concerns: Each service has a dedicated role (polling, saving/processing, and displaying), ensuring loose coupling between components.
- Message-Driven Architecture: RabbitMQ acts as the mediator, decoupling the api-poller and notice-saver, which gives scalability and resilience to message queuing.
- Containerization: All services are isolated within Docker containers, making deployment consistent and environment-independent.
- Modularity and Testability: Notice the presence of tests for each module, suggesting an emphasis on clear module boundaries and unit testing.

## Diagram (Mermaid)

```mermaid
graph TD
    %% External/Infrastructure Components
    Interpol["Interpol API"]:::external
    RabbitMQ["RabbitMQ (Message Broker)"]:::infra
    Database["Database (Persistence)"]:::infra

    %% Docker Compose grouping of services
    subgraph "Docker Compose"
        direction TB
        %% api-poller Service Subgraph
        subgraph "api-poller Service"
            direction TB
            A1["Main Service Code"]:::service
            A2["API Request Session"]:::service
            A3["Query Strategy"]:::service
            A4["RabbitMQ Integration"]:::service
            A5["Configuration Data"]:::config
        end

        %% notice-saver Service Subgraph
        subgraph "notice-saver Service"
            direction TB
            N1["Main Service Code"]:::service
            N2["RabbitMQ Consumer"]:::service
            N3["Image Downloading & Processing"]:::service
            N4["Repository Management"]:::service
            N5["Configuration Data"]:::config
        end

        %% notice-dashboard Service Subgraph
        subgraph "notice-dashboard Service"
            direction TB
            D1["Main Service Code & Entry Point"]:::service
            D2["UI Components & Pages"]:::service
            D3["Database Access & Repositories"]:::service
        end
    end

    %% Data Flow Connections
    Interpol -->|"Polls"| A1
    A4 -->|"Pushes Notices"| RabbitMQ
    RabbitMQ -->|"Delivers Messages"| N2
    N4 -->|"Stores Notices"| Database
    Database -->|"Provides Data"| D3

    %% Click Events for api-poller Service
    click A1 "https://github.com/khanxmetu/interpol-red-dl/tree/main/api-poller/api_poller/"
    click A2 "https://github.com/khanxmetu/interpol-red-dl/blob/main/api-poller/api_poller/api_request_session.py"
    click A3 "https://github.com/khanxmetu/interpol-red-dl/blob/main/api-poller/api_poller/query_strategy.py"
    click A4 "https://github.com/khanxmetu/interpol-red-dl/blob/main/api-poller/api_poller/rabbitmq_client.py"
    click A5 "https://github.com/khanxmetu/interpol-red-dl/blob/main/api-poller/config-data/headers.json"

    %% Click Events for notice-saver Service
    click N1 "https://github.com/khanxmetu/interpol-red-dl/tree/main/notice-saver/notice_saver/"
    click N2 "https://github.com/khanxmetu/interpol-red-dl/blob/main/notice-saver/notice_saver/notice_consumer.py"
    click N3 "https://github.com/khanxmetu/interpol-red-dl/blob/main/notice-saver/notice_saver/image_downloader.py"
    click N4 "https://github.com/khanxmetu/interpol-red-dl/blob/main/notice-saver/notice_saver/repositories_manager.py"
    click N5 "https://github.com/khanxmetu/interpol-red-dl/blob/main/notice-saver/config-data/headers.json"

    %% Click Events for notice-dashboard Service
    click D1 "https://github.com/khanxmetu/interpol-red-dl/blob/main/notice-dashboard/notice_dashboard/Notice_Dashboard.py"
    click D2 "https://github.com/khanxmetu/interpol-red-dl/tree/main/notice-dashboard/notice_dashboard/ui_components/"
    click D3 "https://github.com/khanxmetu/interpol-red-dl/blob/main/notice-dashboard/notice_dashboard/db.py"

    %% Styles
    classDef external fill:#f9c,stroke:#333,stroke-width:2px,color:#000;
    classDef infra fill:#9f9,stroke:#333,stroke-width:2px,color:#000;
    classDef service fill:#cfc,stroke:#333,stroke-width:2px,color:#000;
    classDef config fill:#fc9,stroke:#333,stroke-width:2px,color:#000;
```
