# marshalx/bluesky-feed-generator architecture

> Architecture diagram of the GitHub repository marshalx/bluesky-feed-generator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/marshalx/bluesky-feed-generator
- Repository: https://github.com/marshalx/bluesky-feed-generator

## Overview

##### Main Components

a) External Systems:
- Firehose Data Stream as a separate component
- Client applications that will consume the feed

b) Core Service Components:
- HTTP Server Layer (Flask)
- Feed Generator Service
- Data Stream Processor
- Authentication Handler
- Feed Algorithm Engine
- Data Filter
- Database (SQLite)

##### Component Organization

**[External Clients/Bluesky] (Top Layer)**
- ↕
**[XRPC Endpoints] (API Layer)**
- ↕
**[Feed Generator Core] (Service Layer)**
- ↕
**[Database] (Data Layer)**

##### Specific Implementation Details

a) API Endpoints (show as interface points):
- /.well-known/did.json
- /xrpc/app.bsky.feed.describeFeedGenerator
- /xrpc/app.bsky.feed.getFeedSkeleton

b) Core Components (show as interconnected modules):
- server/app.py (Main Application)
- server/data_stream.py (Firehose Consumer)
- server/algos/feed.py (Feed Algorithm)
- server/data_filter.py (Data Filtering)
- server/database.py (Data Storage)

- Bidirectional communication between the service and Bluesky
- One-way data flow from the Firehose to the Data Stream Processor
- Data flow between the database and service components
- Request/response cycle for feed generation

##### Additional Elements

- A configuration section showing .env and config.py
- A publishing flow for feed registration (publish_feed.py)
- Logging component (logger.py)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Layer
    subgraph External
        BSky[Bluesky/AT Protocol]:::external
        Clients[Client Applications]:::external
        Firehose[Firehose Data Stream]:::external
    end

    %% API Layer
    subgraph API
        DID[".well-known/did.json"]:::api
        Describe["/describeFeedGenerator"]:::api
        GetFeed["/getFeedSkeleton"]:::api
    end

    %% Service Layer
    subgraph Service
        Server["HTTP Server"]:::core
        Auth["Authentication Handler"]:::security
        StreamProc["Data Stream Processor"]:::core
        FeedAlgo["Feed Algorithm Engine"]:::core
        DataFilter["Data Filter"]:::core
        Config["Configuration System"]:::config
        Logger["Logging System"]:::utility
        Publisher["Feed Publisher"]:::utility
    end

    %% Data Layer
    DB[(SQLite Database)]:::database

    %% Connections
    BSky <--> Server
    Firehose --> StreamProc
    Clients --> BSky
    
    Server --> DID
    Server --> Describe
    Server --> GetFeed
    
    Server <--> Auth
    Server <--> FeedAlgo
    StreamProc --> DataFilter
    DataFilter --> DB
    FeedAlgo <--> DB
    
    Config --> Server
    Config --> StreamProc
    Logger --> Server
    Publisher --> BSky

    %% Component Click Events
    click Server "https://github.com/MarshalX/bluesky-feed-generator/blob/main/server/app.py"
    click Auth "https://github.com/MarshalX/bluesky-feed-generator/blob/main/server/auth.py"
    click FeedAlgo "https://github.com/MarshalX/bluesky-feed-generator/blob/main/server/algos/feed.py"
    click DataFilter "https://github.com/MarshalX/bluesky-feed-generator/blob/main/server/data_filter.py"
    click StreamProc "https://github.com/MarshalX/bluesky-feed-generator/blob/main/server/data_stream.py"
    click DB "https://github.com/MarshalX/bluesky-feed-generator/blob/main/server/database.py"
    click Logger "https://github.com/MarshalX/bluesky-feed-generator/blob/main/server/logger.py"
    click Config "https://github.com/MarshalX/bluesky-feed-generator/blob/main/server/config.py"
    click Publisher "https://github.com/MarshalX/bluesky-feed-generator/blob/main/publish_feed.py"

    %% Styling
    classDef external fill:#a8d1f0,stroke:#333
    classDef api fill:#90EE90,stroke:#333
    classDef core fill:#FFE5B4,stroke:#333
    classDef security fill:#FFB6C1,stroke:#333
    classDef database fill:#E6E6E6,stroke:#333
    classDef config fill:#DDA0DD,stroke:#333
    classDef utility fill:#F0E68C,stroke:#333

    %% Legend
    subgraph Legend
        L1[External Services]:::external
        L2[API Endpoints]:::api
        L3[Core Components]:::core
        L4[Security]:::security
        L5[(Database)]:::database
        L6[Configuration]:::config
        L7[Utility Services]:::utility
    end
```
