# olist/olist-loafer architecture

> Architecture diagram of the GitHub repository olist/olist-loafer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/olist/olist-loafer
- Repository: https://github.com/olist/olist-loafer

## Overview

##### Main components
- Core Engine: This is represented by modules within src/loafer such as the LoaferManager, dispatchers, runners, and managers. They coordinate the flow of messages and manage the lifecycle of asynchronous processing.
- Routes: Found in src/loafer/routes.py and src/loafer/ext/aws/routes.py, these define how message sources (queues) are associated with handlers.
- Handlers: Modules like src/loafer/ext/aws/handlers.py implement the logic for processing messages (e.g., transforming and forwarding messages from a source queue to a destination queue).
- Message Translators: Components in src/loafer/message_translators.py and its AWS counterpart, which facilitate the transformation between message formats.
- Providers/Integrations: The ext/aws directory provides AWS-specific integration components – SQS, via providers and bases – and there is also integration with Sentry (error handling) as seen in src/loafer/ext/sentry.py.
- Exceptions & Compat: The project has a dedicated exception module to manage errors and ensure decoupling, aiding in a robust and extensible design.

##### Recognize architectural relationships
- The LoaferManager is the central coordinator that ties the configured routes to their specific handlers.
- Each route (e.g., SQSRoute) links a source (like 'source-queue') to a specific handler (e.g., SQSHandler for 'destination-queue'). This clearly indicates dependency and data flow from AWS SQS service to the internal processing routines.
- Dispatchers and runners orchestrate asynchronous processing by managing tasks concurrently.
- There is clear separation between framework infrastructure (in core modules) and provider-specific implementations (under the ext/aws directory).

##### Architectural patterns and design principles
- Extensibility/Customization: Note that the code is designed to be extended (for example, support for multiple providers, error handlers, and various routes). Represent extension interfaces as plug-in modules.
- Asynchronous Processing: Since the project is built for asynchronous message handling, emphasize the communication flow with self-contained asynchronous components (use “async” symbols or notations if available).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Services
    AWS_Source["AWS SQS (Source Queue)"]:::external

    %% Routes Layer
    subgraph "Routes"
        RoutesGen["Routes (General)"]:::route
        RoutesAWS["AWS Routes"]:::route
    end

    %% Handlers Layer
    subgraph "Handlers"
        AWS_Handler["AWS Handler"]:::aws
    end

    %% Message Translators Layer
    subgraph "Message Translators"
        TranslatorGen["Translator (General)"]:::translator
        TranslatorAWS["AWS Translator"]:::translator
    end

    %% Core Engine Layer
    subgraph "Core Engine"
        LoaferManager["LoaferManager: Main Orchestrator"]:::core
        Dispatchers["Dispatchers: Task Manager"]:::core
        Runners["Runners: Async Runner"]:::core
    end

    %% Providers / Integrations
    subgraph "Providers/Integrations"
        AWS_Providers["AWS Providers"]:::aws
        AWS_Bases["AWS Bases"]:::aws
    end

    %% External Output
    subgraph "External Services"
        AWS_Destination["AWS SQS (Destination Queue)"]:::external
    end

    %% Error Monitoring
    subgraph "Error Monitoring"
        Sentry_Monitor["Sentry Monitoring"]:::monitor
    end

    %% Exceptions & Compatibility
    subgraph "Exceptions & Compat"
        Exceptions["Exceptions Module"]:::exception
        Compat_Layer["Compat Layer"]:::exception
    end

    %% Flow Connections
    AWS_Source -->|"async"| RoutesGen
    AWS_Source -->|"async"| RoutesAWS

    RoutesGen -->|"defines"| AWS_Handler
    RoutesAWS -->|"defines"| AWS_Handler

    AWS_Handler -->|"calls"| TranslatorGen
    AWS_Handler -->|"calls"| TranslatorAWS

    TranslatorGen -->|"translates"| LoaferManager
    TranslatorAWS -->|"translates"| LoaferManager

    LoaferManager -->|"coordinates"| Dispatchers
    LoaferManager -->|"coordinates"| Runners

    LoaferManager -->|"triggers"| AWS_Providers
    Dispatchers -->|"triggers"| AWS_Bases

    AWS_Providers -->|"sends"| AWS_Destination
    AWS_Bases -->|"sends"| AWS_Destination

    LoaferManager -->|"error"| Sentry_Monitor
    Runners -->|"error"| Sentry_Monitor

    %% Click Events for Core Engine
    click LoaferManager "https://github.com/olist/olist-loafer/blob/main/src/loafer/managers.py"
    click Dispatchers "https://github.com/olist/olist-loafer/blob/main/src/loafer/dispatchers.py"
    click Runners "https://github.com/olist/olist-loafer/blob/main/src/loafer/runners.py"

    %% Click Events for Routes
    click RoutesGen "https://github.com/olist/olist-loafer/blob/main/src/loafer/routes.py"
    click RoutesAWS "https://github.com/olist/olist-loafer/blob/main/src/loafer/ext/aws/routes.py"

    %% Click Event for Handler
    click AWS_Handler "https://github.com/olist/olist-loafer/blob/main/src/loafer/ext/aws/handlers.py"

    %% Click Events for Message Translators
    click TranslatorGen "https://github.com/olist/olist-loafer/blob/main/src/loafer/message_translators.py"
    click TranslatorAWS "https://github.com/olist/olist-loafer/blob/main/src/loafer/ext/aws/message_translators.py"

    %% Click Events for Providers/Integrations
    click AWS_Providers "https://github.com/olist/olist-loafer/blob/main/src/loafer/ext/aws/providers.py"
    click AWS_Bases "https://github.com/olist/olist-loafer/blob/main/src/loafer/ext/aws/bases.py"

    %% Click Event for Error Monitoring
    click Sentry_Monitor "https://github.com/olist/olist-loafer/blob/main/src/loafer/ext/sentry.py"

    %% Click Events for Exceptions & Compat
    click Exceptions "https://github.com/olist/olist-loafer/blob/main/src/loafer/exceptions.py"
    click Compat_Layer "https://github.com/olist/olist-loafer/blob/main/src/loafer/_compat.py"

    %% Styles
    classDef external fill:#ffebcc,stroke:#f39c12,stroke-width:2px;
    classDef core fill:#d5f5e3,stroke:#27ae60,stroke-width:2px;
    classDef aws fill:#fadbd8,stroke:#e74c3c,stroke-width:2px;
    classDef translator fill:#d6eaf8,stroke:#3498db,stroke-width:2px;
    classDef route fill:#e8daef,stroke:#8e44ad,stroke-width:2px;
    classDef monitor fill:#f9e79f,stroke:#f1c40f,stroke-width:2px;
    classDef exception fill:#fbeee6,stroke:#e67e22,stroke-width:2px;
```
