# nanos/fedifetcher architecture

> Architecture diagram of the GitHub repository nanos/fedifetcher, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nanos/fedifetcher
- Repository: https://github.com/nanos/fedifetcher

## Overview

##### Main Components

a) Client Components:
- Multiple execution methods (represent as parallel entry points):
- GitHub Actions runner
- Cron job
- Container runtime
- Windows Task Scheduler

b) Core Components:
- Main Script Modules:
- find_posts.py (Primary execution module)
- get_context.py (Context fetching module)

c) External Systems:
- Mastodon API (Primary integration point)
- Other Fediverse Platforms:
- Pleroma/Akkoma
- Misskey/Calckey/Firefish
- Lemmy
- Peertube

d) Storage:
- Artifacts storage (for caching fetched posts)

**[Client Layer]**
- ↓
**[Core Processing Layer]**
- ↓
**[External Integration Layer]**
- ↓
**[Storage Layer]**

##### Component Details

For each component, show:
- Input/output relationships
- Key functionalities:
- Timeline fetching
- Reply fetching
- Profile backfilling
- Cache management

##### Technical Details

A technical metadata section showing:
- Python runtime requirement
- Docker container support
- API version compatibility
- Required permissions/scopes

##### The configurable nature of the fetching operations

Keep the abstraction level focused on the system architecture while including enough detail to show the unique aspects of FediFetcher, particularly its various fetching capabilities and caching mechanism.

Remember to indicate the asynchronous nature of the operations and the potential for long-running processes, as mentioned in the README's warning about execution time.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Client Layer"
        GHA["GitHub Actions"]
        CRON["Cron Job"]
        CONTAINER["Container Runtime"]
        WTASK["Windows Task Scheduler"]
    end

    subgraph "Core Processing Layer"
        FIND["Find Posts Module"]
        CONTEXT["Context Module"]
        CONFIG["Configuration"]
    end

    subgraph "External Integration Layer"
        MASTODON["Mastodon API"]
        PLEROMA["Pleroma/Akkoma"]
        MISSKEY["Misskey/Calckey/Firefish"]
        LEMMY["Lemmy"]
        PEERTUBE["Peertube"]
    end

    subgraph "Storage Layer"
        ARTIFACTS["Artifacts Storage"]
    end

    %% Client to Core connections
    GHA --> FIND
    CRON --> FIND
    CONTAINER --> FIND
    WTASK --> FIND

    %% Core internal connections
    FIND <--> CONTEXT
    FIND --> CONFIG
    CONTEXT --> CONFIG

    %% Core to External connections
    FIND --> MASTODON
    FIND -.-> PLEROMA
    FIND -.-> MISSKEY
    FIND -.-> LEMMY
    FIND -.-> PEERTUBE

    %% Storage connections
    FIND <--> ARTIFACTS
    CONTEXT <--> ARTIFACTS

    %% Styling
    classDef client fill:#FFE873
    classDef core fill:#6495ED
    classDef external fill:#90EE90
    classDef storage fill:#D3D3D3
    classDef config fill:#FFB6C1

    class GHA,CRON,CONTAINER,WTASK client
    class FIND,CONTEXT core
    class MASTODON,PLEROMA,MISSKEY,LEMMY,PEERTUBE external
    class ARTIFACTS storage
    class CONFIG config

    %% Click events
    click FIND "https://github.com/nanos/FediFetcher/blob/main/find_posts.py"
    click CONTEXT "https://github.com/nanos/FediFetcher/blob/main/get_context.py"
    click ARTIFACTS "https://github.com/nanos/FediFetcher/tree/main/artifacts/"
    click CONTAINER "https://github.com/nanos/FediFetcher/tree/main/Dockerfile"
    click GHA "https://github.com/nanos/FediFetcher/blob/main/.github/workflows/get_context.yml"

    %% Legend
    subgraph Legend
        L1["Client Component"]:::client
        L2["Core Component"]:::core
        L3["External Service"]:::external
        L4["Storage"]:::storage
        L5["Configuration"]:::config
        L6[" --> Direct Flow"]
        L7[" -.-> Optional Flow"]
    end
```
