# hitomi-team/shimeji architecture

> Architecture diagram of the GitHub repository hitomi-team/shimeji, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hitomi-team/shimeji
- Repository: https://github.com/hitomi-team/shimeji

## Overview

##### Project type and purpose
- This is a Python library (“shimeji”) for building open-domain chatbots in a platform- and API-agnostic way.
- Key indicators:
- `shimeji/` package with modules like preprocessor, postprocessor, model_provider, memory, etc.
- Alembic migration config (`alembic.ini`, `alembic/versions/…`) for a memory table.
- README describes “A platform and API Agnostic library for writing Open Domain and General Chatbots.”

##### File structure
Top-level directories/files:
- `shimeji/` – core Python package
– preprocessor.py, postprocessor.py: pipeline stages for cleaning or shaping user input/output
– model_provider.py: plugin interface to LLM APIs or local models
– memory.py + memorystore_provider.py + sqlcrud.py: in-process memory and memory persistence layer
– util.py: helper routines
– shimeji.py: orchestrator that ties all modules into a chatbot pipeline
- `alembic/` + `alembic.ini`: database migration setup for persistent memory (SQLAlchemy + Alembic)
- `docs/`: Sphinx documentation source
- `requirements.txt` + `setup.py`: packaging and dependency declarations
- `test.py`, `test_async.py`: example or test scripts

##### Architecture insights from README
- Persistence (SQL) is optional but pluggable via `memorystore_provider`.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client Application"
        Client["Client Application"]:::client
    end

    subgraph "Shimeji Library Core"
        Orchestrator["Shimeji Orchestrator"]:::core
        Preprocessor["Preprocessor"]:::core
        MemoryManager["Memory Manager\n(in-memory)"]:::core
        MemProvider["Persistent Memory Store Provider"]:::core
        SQLAdapter["SQL CRUD Adapter"]:::persistence
        ModelProvider["Model Provider Interface"]:::core
        Postprocessor["Postprocessor"]:::core
        Util["Utilities"]:::util
    end

    subgraph "Persistence Layer"
        DB[(SQL Database)]:::db
        AlembicConfig["Alembic Config"]:::persistence
        Migration["Memory Table Migration"]:::persistence
    end

    subgraph "External Services"
        LLM["External LLM API"]:::external
    end

    subgraph "Documentation"
        Docs["Sphinx Docs"]:::util
    end

    subgraph "Packaging & Dependencies"
        PKG["requirements.txt\nsetup.py"]:::util
    end

    Client -->|"invoke"| Orchestrator
    Orchestrator -->|"text_in"| Preprocessor
    Preprocessor -->|"cleaned_text"| MemoryManager
    MemoryManager -->|"state"| MemProvider
    MemProvider -->|"persisted_state"| SQLAdapter
    SQLAdapter -->|"CRUD ops"| DB
    AlembicConfig -->|"migrations"| DB
    Migration -->|"migration script"| DB
    MemoryManager -->|"in-memory state"| ModelProvider
    ModelProvider -->|"api_request"| LLM
    LLM -->|"api_response"| ModelProvider
    ModelProvider -->|"response"| Postprocessor
    Postprocessor -->|"text_out"| Orchestrator
    Orchestrator -->|"return"| Client
    Orchestrator --> Util
    SQLAdapter --> Util

    click Client "https://github.com/hitomi-team/shimeji/blob/main/test.py"
    click Orchestrator "https://github.com/hitomi-team/shimeji/blob/main/shimeji/shimeji.py"
    click Preprocessor "https://github.com/hitomi-team/shimeji/blob/main/shimeji/preprocessor.py"
    click Postprocessor "https://github.com/hitomi-team/shimeji/blob/main/shimeji/postprocessor.py"
    click MemoryManager "https://github.com/hitomi-team/shimeji/blob/main/shimeji/memory.py"
    click MemProvider "https://github.com/hitomi-team/shimeji/blob/main/shimeji/memorystore_provider.py"
    click SQLAdapter "https://github.com/hitomi-team/shimeji/blob/main/shimeji/sqlcrud.py"
    click AlembicConfig "https://github.com/hitomi-team/shimeji/blob/main/alembic.ini"
    click Migration "https://github.com/hitomi-team/shimeji/blob/main/alembic/versions/0b35c351f36e_create_memory_table.py"
    click ModelProvider "https://github.com/hitomi-team/shimeji/blob/main/shimeji/model_provider.py"
    click Util "https://github.com/hitomi-team/shimeji/blob/main/shimeji/util.py"
    click Docs "https://github.com/hitomi-team/shimeji/tree/main/docs/"
    click PKG "https://github.com/hitomi-team/shimeji/blob/main/requirements.txt"

    classDef core fill:#D0E8FF,stroke:#333,stroke-width:1px
    classDef persistence fill:#D0FFD6,stroke:#333,stroke-width:1px
    classDef external fill:#FFE0B2,stroke:#333,stroke-width:1px
    classDef util fill:#E0E0E0,stroke:#333,stroke-width:1px
    classDef db fill:#C8FFD6,stroke:#333,stroke-width:1px
    classDef client fill:#E8D0FF,stroke:#333,stroke-width:1px
```
