# johnconnor-sec/fabric architecture

> Architecture diagram of the GitHub repository johnconnor-sec/fabric, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/johnconnor-sec/fabric
- Repository: https://github.com/johnconnor-sec/fabric

## Overview

##### Project type and purpose
- This is a full-stack, open-source AI-augmentation framework (“fabric”) written in Go, with:
- A command-line interface (CLI) for invoking “Patterns” (prompt templates)
- A plugin system for AI providers, databases, and template engines
- A REST API server that exposes pattern execution over HTTP
- A Svelte-based web GUI
- A Streamlit UI for rapid prototyping

##### File structure
- Root contains Dockerfile, docker-compose.yml, main.go, and version.go → the entrypoint for both CLI and API server.
- /cli → Go code for command parsing, flags, and CLI commands.
- /common → Shared domain types and utilities.
- /core → Core engine: “chatter” orchestrates pattern execution; plugin_registry loads AI, DB, template plugins.
- /plugins → Four subpackages:
- ai → adapters for OpenAI, Anthropic, Gemini, etc.
- db → file-system DB (fsdb) for contexts, patterns, sessions.
- template → text/template extensions and custom functions.
- tools → helper apps (e.g., to_pdf).
- /restapi → HTTP handlers for chat, patterns, sessions, storage and exposes “fabric --serve”.
- /patterns → Markdown-based pattern definitions (system.md, user.md) organized by pattern name.
- /web → Svelte front-end:
- /src/lib/api → REST client wrappers
- /src/lib/components → UI components (Chat, Patterns, Posts, Settings)
- /src/lib/services and /src/lib/store → state management and business logic
- /routes → page endpoints served by SvelteKit
- streamlit.py → Python Streamlit UI alternative

##### Extract insights from README
- Patterns are central: loaded from disk via the DB plugin, executed by core.chatter → plugin_registry → AI plugins → returns structured Markdown.
- The web GUI uses `fabric --serve` (REST API) to power chat and pattern browsing/editing.
- CLI, Web UI, and Streamlit share the same core engine and plugin system.

##### Main components
A. Clients
- CLI executable (fabric)
- REST API consumers (Svelte web GUI, Streamlit UI, external HTTP clients)
- Helper binaries (to_pdf, other tools)
B. Core Engine
- main.go → initializes CLI or API server
- core.chatter → orchestrates pattern execution
- plugin_registry → dynamically discovers and loads plugins
C. Plugins
- AI providers (OpenAI, Anthropic, Gemini, etc.)
- DB plugin (fsdb) for storing patterns, contexts, sessions
- Template plugin for text transformations
- Tools plugin for command-line utilities
D. Patterns Storage
- /patterns directory (built-in)
- User patterns in ~/.config/fabric/patterns
E. Data Stores
- File system (via fsdb)
F. External Services
- LLM APIs (OpenAI, Azure, Anthropic…)
- YouTube API (for transcripts)
- Obsidian or other MD vaults (optional)

##### Relationships & data flow
1. User invokes CLI or Web/Streamlit → passes pattern name + input
2. main.go routes to core.chatter
3. core.chatter loads the pattern from fsdb via plugin_registry
4. core.chatter calls AI plugin(s) to execute the prompt
5. AI plugin communicates with external LLM service
6. Response flows back through core → CLI writes to stdout / file OR REST API returns JSON
7. Web GUI/Streamlit consumes REST endpoints to display and manage sessions
8. Tools plugin (e.g., to_pdf) can be chained on CLI output

##### Architectural patterns & design principles
- Plugin Architecture: decouples core engine from AI, database, template, and tool implementations
- Modular layering:
- Presentation layer (CLI, REST API, Web, Streamlit)
- Application layer (core.chatter, plugin_registry)
- Infrastructure layer (plugins for AI, DB, templates)
- Configuration-as-code: patterns are Markdown files, discovered at runtime
- Single-codebase polyglot front-end: SvelteKit for web, Python for Streamlit, Go for core

##### Relevant technologies & frameworks
- Go: CLI, core engine, REST server
- fsdb (Go-based file-system DB)
- SvelteKit + TypeScript + Tailwind: web GUI
- Python + Streamlit: alternate UI
- Docker/Nix: containerization & reproducible builds

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Presentation Layer
    subgraph "Presentation Layer"
        direction TB
        CLI["CLI Executable"]:::ui
        WebGUI["Web GUI (SvelteKit)"]:::ui
        Streamlit["Streamlit UI"]:::ui
        ExternalHTTP["External HTTP Clients"]:::ui
    end

    %% Application Layer
    subgraph "Application Layer"
        direction TB
        Main["main.go / version.go"]:::core
        Chatter["core.chatter"]:::core
        PluginRegistry["plugin_registry"]:::core
        RESTAPI["REST API Server"]:::core
    end

    %% Infrastructure Layer
    subgraph "Plugins"
        direction TB
        AIPlugins["AI Providers"]:::plugin
        DBPlugin["DB Plugin (fsdb)"]:::plugin
        TemplatePlugin["Template Plugin"]:::plugin
        ToolsPlugin["Tools Plugin"]:::plugin
    end

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        PatternsFS["Patterns Directory"]:::store
        FSDB["fsdb Storage"]:::store
    end

    %% External Services
    subgraph "External Services"
        direction TB
        LLM["LLM APIs (OpenAI/Azure/Anthropic)"]:::external
        YouTube["YouTube API"]:::external
        Obsidian["Obsidian MD Vault"]:::external
    end

    %% Connections
    CLI -->|"invokes"| Main
    WebGUI -->|"calls"| RESTAPI
    Streamlit -->|"calls"| RESTAPI
    ExternalHTTP -->|"calls"| RESTAPI

    Main -->|"/serve or CLI"| RESTAPI
    Main -->|"routes"| Chatter
    Chatter -->|"loads patterns"| DBPlugin
    DBPlugin -->|"reads/writes"| FSDB
    DBPlugin -->|"reads"| PatternsFS
    Chatter -->|"executes prompt"| AIPlugins
    AIPlugins -->|"calls"| LLM
    Chatter -->|"uses"| TemplatePlugin
    CLI -->|"can chain"| ToolsPlugin
    ToolsPlugin -->|"optional"| FSDB

    RESTAPI -->|"uses"| Chatter
    WebGUI -->|"serves UI from"| web
    Streamlit -->|"runs"| streamlit

    %% Click events for component mapping
    click CLI "https://github.com/johnconnor-sec/fabric/blob/main/main.go"
    click CLI "https://github.com/johnconnor-sec/fabric/blob/main/version.go"
    click CLI "https://github.com/johnconnor-sec/fabric/blob/main/cli/cli.go"
    click CLI "https://github.com/johnconnor-sec/fabric/blob/main/cli/flags.go"
    click CLI "https://github.com/johnconnor-sec/fabric/blob/main/cli/output.go"
    click Chatter "https://github.com/johnconnor-sec/fabric/blob/main/core/chatter.go"
    click PluginRegistry "https://github.com/johnconnor-sec/fabric/blob/main/core/plugin_registry.go"
    click AIPlugins "https://github.com/johnconnor-sec/fabric/tree/main/plugins/ai/"
    click AIPlugins "https://github.com/johnconnor-sec/fabric/blob/main/plugins/ai/openai/openai.go"
    click AIPlugins "https://github.com/johnconnor-sec/fabric/blob/main/plugins/ai/anthropic/anthropic.go"
    click AIPlugins "https://github.com/johnconnor-sec/fabric/blob/main/plugins/ai/gemini/gemini.go"
    click AIPlugins "https://github.com/johnconnor-sec/fabric/blob/main/plugins/ai/models.go"
    click DBPlugin "https://github.com/johnconnor-sec/fabric/tree/main/plugins/db/fsdb/"
    click DBPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/db/fsdb/db.go"
    click DBPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/db/fsdb/contexts.go"
    click DBPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/db/fsdb/patterns.go"
    click DBPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/db/fsdb/sessions.go"
    click DBPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/db/fsdb/storage.go"
    click TemplatePlugin "https://github.com/johnconnor-sec/fabric/tree/main/plugins/template/"
    click TemplatePlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/template/template.go"
    click TemplatePlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/template/extension_executor.go"
    click TemplatePlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/template/extension_manager.go"
    click TemplatePlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/template/extension_registry.go"
    click ToolsPlugin "https://github.com/johnconnor-sec/fabric/tree/main/plugins/tools/"
    click ToolsPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/tools/to_pdf/to_pdf.go"
    click ToolsPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/tools/patterns_loader.go"
    click ToolsPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/tools/converter/html_readability.go"
    click ToolsPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/tools/jina/jina.go"
    click ToolsPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/tools/lang/language.go"
    click ToolsPlugin "https://github.com/johnconnor-sec/fabric/blob/main/plugins/tools/youtube/youtube.go"
    click PatternsFS "https://github.com/johnconnor-sec/fabric/tree/main/patterns/"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/tree/main/restapi/"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/blob/main/restapi/serve.go"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/blob/main/restapi/chat.go"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/blob/main/restapi/patterns.go"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/blob/main/restapi/sessions.go"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/blob/main/restapi/contexts.go"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/blob/main/restapi/storage.go"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/blob/main/restapi/models.go"
    click RESTAPI "https://github.com/johnconnor-sec/fabric/blob/main/restapi/configuration.go"
    click WebGUI "https://github.com/johnconnor-sec/fabric/tree/main/web/"
    click WebGUI "https://github.com/johnconnor-sec/fabric/tree/main/web/src/lib/api/"
    click WebGUI "https://github.com/johnconnor-sec/fabric/tree/main/web/src/lib/components/"
    click WebGUI "https://github.com/johnconnor-sec/fabric/tree/main/web/src/lib/services/"
    click WebGUI "https://github.com/johnconnor-sec/fabric/tree/main/web/src/lib/store/"
    click WebGUI "https://github.com/johnconnor-sec/fabric/tree/main/web/src/routes/"
    click Streamlit "https://github.com/johnconnor-sec/fabric/blob/main/streamlit.py"
    click FSDB "https://github.com/johnconnor-sec/fabric/tree/main/plugins/db/fsdb/"
    click FSDB "https://github.com/johnconnor-sec/fabric/tree/main/patterns/"

    %% Styles
    classDef ui fill:#FFA500,stroke:#333,stroke-width:1px
    classDef core fill:#7FB3D5,stroke:#333,stroke-width:1px
    classDef plugin fill:#B2BABB,stroke:#333,stroke-width:1px
    classDef store fill:#82E0AA,stroke:#333,stroke-width:1px
    classDef external fill:#D7DBDD,stroke:#333,stroke-width:1px
```
