# zdenek-valehrach/segmenstory architecture

> Architecture diagram of the GitHub repository zdenek-valehrach/segmenstory, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zdenek-valehrach/segmenstory
- Repository: https://github.com/zdenek-valehrach/segmenstory

## Overview

##### Project type and purpose
- SegmenStory is a monolithic, front-end-centric “full-stack” application built with Streamlit. It binds a simple UI (Streamlit app) to two external AI services:
- Mask2Former segmentation via Hugging Face API
- Generative text (Perplexity “sonar” or OpenAI gpt-4o)
- There is no persistent database; all processing happens at runtime in a single Python process.

##### File structure
- Top‐level:
- app.py – Streamlit UI entry point
- config.py – defaults for all API keys and parameters
- models/
- segmentation.py – wrapper for Mask2Former calls and mask decoding
- llm.py – wrapper for Perplexity/OpenAI text generation
- utils/
- image_utils.py – resizing, format conversion, mask overlay
- prompt_utils.py – prompt‐building logic
- coco_class_map.py – Czech translation and normalization of COCO class names
- .devcontainer – environment definition

##### README for additional insights
- Describes stepwise workflow: image upload → segmentation → mask visualization → segment selection → prompt generation → LLM call → story display
- Highlights key modules you already see in the tree

- Streamlit UI (app.py + Streamlit runtime)
- Core Processing modules:
– Segmentation Module (models/segmentation.py + image_utils)
– LLM Module (models/llm.py + prompt_utils + coco_class_map)
- Configuration Manager (config.py / sidebar input)
– Hugging Face Mask2Former API
– Perplexity API
– OpenAI API
1. User → Streamlit UI: upload image, enter API keys, choose provider/topic
2. UI → Segmentation Module: raw image → resized image → HF API call
4. UI → LLM Module: selected segment class + optional user topic → prompt_utils + coco_class_map → built prompt
5. LLM Module → chosen API (Perplexity or OpenAI) → raw story → artifact cleaner → UI for display
6. Config Manager → provides API keys and default params to both modules
c. Architectural patterns & principles:
- Modular monolith with clear separation of concerns (UI, segmentation, LLM, utils, config)
- Pipeline pattern: data flows through discrete stages
- Dependency inversion: high-level modules (app.py) depend on abstractions (wrappers in models/)
d. Key technologies & frameworks:
- Python 3.x, Streamlit for UI
- Hugging Face REST API (Mask2Former)
- OpenAI REST API, Perplexity REST API
- PIL, NumPy for image processing
- Base64 encoding/decoding for masks

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction
    User["User"]:::external
    User -->|"Image upload, API keys, topic"| UI

    %% Internal Modules
    subgraph "Streamlit UI" 
        direction TB
        UI["Streamlit UI\n(app.py, Streamlit runtime)"]:::internal
    end
    subgraph "Configuration Manager"
        direction TB
        Config["Configuration Manager\n(config.py, sidebar inputs)"]:::config
    end
    subgraph "Segmentation Module"
        direction TB
        SegCore["Segmentation Core\n(models/segmentation.py)"]:::internal
        ImgUtils["Image Utils\n(utils/image_utils.py)"]:::internal
    end
    subgraph "LLM Module"
        direction TB
        LLW["LLM Wrapper\n(models/llm.py)"]:::internal
        PromptUtils["Prompt Utils\n(utils/prompt_utils.py)"]:::internal
        CocoMap["COCO Class Map\n(utils/coco_class_map.py)"]:::internal
    end

    %% External Services
    HF["Hugging Face Mask2Former API"]:::external
    Perplexity["Perplexity API"]:::external
    OpenAI["OpenAI API"]:::external

    %% Development Environment
    DevEnv["Dev Container\n(.devcontainer/devcontainer.json)"]:::devenv

    %% Data Flows
    UI -->|"Raw image"| SegCore
    SegCore -->|"Resized image"| HF
    HF -->|"Base64 masks"| ImgUtils
    ImgUtils -->|"Colored overlays"| UI

    UI -->|"Selected segment + topic"| LLW
    LLW -->|"Built prompt"| PromptUtils
    PromptUtils -->|"Normalized classes"| CocoMap
    CocoMap -->|"Final prompt"| LLW
    LLW -->|"API call"| Perplexity
    LLW -->|"API call"| OpenAI
    Perplexity -->|"Raw story"| LLW
    OpenAI -->|"Raw story"| LLW
    LLW -->|"Cleaned story"| UI

    %% Configuration Dependencies
    Config --> UI
    Config --> SegCore
    Config --> LLW

    %% Legend
    subgraph Legend
        direction TB
        A[Blue = Internal Modules]:::internal
        B[Green = External Services]:::external
        C[Orange = Configuration]:::config
        D[Grey = Dev Env]:::devenv
    end

    %% Click Events
    click UI "https://github.com/zdenek-valehrach/segmenstory/blob/main/app.py"
    click Config "https://github.com/zdenek-valehrach/segmenstory/blob/main/config.py"
    click SegCore "https://github.com/zdenek-valehrach/segmenstory/blob/main/models/segmentation.py"
    click ImgUtils "https://github.com/zdenek-valehrach/segmenstory/blob/main/utils/image_utils.py"
    click LLW "https://github.com/zdenek-valehrach/segmenstory/blob/main/models/llm.py"
    click PromptUtils "https://github.com/zdenek-valehrach/segmenstory/blob/main/utils/prompt_utils.py"
    click CocoMap "https://github.com/zdenek-valehrach/segmenstory/blob/main/utils/coco_class_map.py"
    click DevEnv "https://github.com/zdenek-valehrach/segmenstory/blob/main/.devcontainer/devcontainer.json"

    %% Styles
    classDef internal fill:#D0E6FF,stroke:#0366D6,color:#0366D6;
    classDef external fill:#E6FFEA,stroke:#28A745,color:#28A745,stroke-dasharray: 5 5;
    classDef config fill:#FFF3CD,stroke:#FFCD39,color:#856404;
    classDef devenv fill:#F0F0F0,stroke:#CCCCCC,color:#666666;
```
