# fabriclabs/sensemaker architecture

> Architecture diagram of the GitHub repository fabriclabs/sensemaker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fabriclabs/sensemaker
- Repository: https://github.com/fabriclabs/sensemaker

## Overview

##### Major Components
- Frontend:
– React components (under the “components” directory) and associated assets (in “assets”) form the interactive user interface.
– Redux actions and reducers (directories “actions” and “reducers”) managing state.
- Backend:
– Node.js server scripts including “scripts/node.js” and the several route definitions (in “routes”) that handle HTTP endpoints.
– Services (in “services”, for example “sensemaker.js”, email, fabric, etc.) implement business logic and interface with external systems.
- Data Layers:
– MySQL is used as the primary relational database (as indicated by the “knexfile.js”, migration files in “migrations”, and MySQL requirement).
– Redis (with its designated “redisActions.js” and “routes/redis”) is used for caching, queue management, and potentially session handling.
- External/Integration Layers:
– The Fabric Network is a key element. This is seen in both the README (“Fabric” section) and in integrations with external modules such as “@fabric/core”, “@fabric/http”, and “@fabric/hub”. This external network stream is used to collect data.
– Additional services (e.g., services for chat, email, blockchain actions in “actions/bitcoinActions.js”) might connect to external APIs or systems.
- Utility & Infrastructure:
– There are utility scripts (in “scripts”), helper functions (in “functions”), and tests (in “tests”) supporting operational tasks, build processes, and validations.
– Dockerfile, CI/CD configurations (.github/workflows, .travis.yml) and deployment scripts indicate a modern, containerized, and automated deployment strategy.

##### Relationships and Data Flows
- The end users interact with the Frontend (React UI). Their actions trigger Redux actions, which in turn may initiate API calls.
- The API calls are routed through the Node.js backend (via “routes” and “controllers”) where business logic is applied.
- Backend services (inside “services” and “actions”, plus possibly “functions”) perform data processing, especially the LLM-based pipeline and aggregate information from the Fabric Network.
- The backend communicates with the MySQL database for persistent storage (schema defined in migrations) and with Redis for caching/queue management.
- External integration with Fabric and possibly other networks (e.g., Discord, GitHub, blockchain via “bitcoinActions”) is maintained via dedicated service modules.
- Utility scripts and scheduled tasks (from “scripts”, “functions”, “migrations”, and “tests”) ensure system health and scalability.

##### Architectural Patterns and Key Technologies
- MVC / RESTful API Pattern: Routes serve as controllers, while the “actions” and “reducers” patterns follow a Redux-like state management approach.
- Microservice/Modular Services Design: The existence of many discrete “services” (sensemaker, fabric, email, stripe, etc.) and specialized routes shows modularity.
- Integration of LLM Pipeline: Multi-stage processing likely occurs across actions, services, and even through external processing (the “LLM Pipeline” in README).
- Technologies:
- Node.js (version 18.19.1) for the backend.
- React/Redux for the frontend.
- MySQL (8.3) and Redis Stack (7.2) for data and caching.
- Fabric Network libraries for data aggregation.
- Additional libraries (like the extensive Semantic UI library under “libraries/semantic”) for styling and UI behavior.

##### Instructing the Principal Software Engineer
1. Client Tier (UI components + state management)
2. Application Tier (Node.js API, route handling, business logic, LLM pipeline)
3. Data Tier (MySQL, Redis)
4. Integration Tier (Fabric Network, external services)
- User interactions initiated on the client and routed to the API.
- Backend processing which may call external services and then update the database/cache.
- Feedback loops such as asynchronous tasks (e.g., message queues via Redis) or notifications to the frontend.
- Do not shy away from breaking down the system into as many logical components as possible – separation of concerns is critical given the complexity of actions, routes, and services.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Tier
    subgraph "Client Tier"
        FE["React Frontend"]:::client
        RA["Redux Actions"]:::client
        RR["Redux Reducers"]:::client
        AS["Static Assets"]:::client
    end

    %% Application Tier
    subgraph "Application Tier"
        NS["Node.js Server"]:::apptier
        AR["API Routes"]:::apptier
        BL["Business Logic Services (including LLM Pipeline)"]:::apptier
    end

    %% Data Tier
    subgraph "Data Tier"
        MY["MySQL Database"]:::datatier
        RC["Redis Cache"]:::datatier
    end

    %% Integration Tier
    subgraph "Integration Tier"
        FI["Fabric Integration Service"]:::integ
        BTC["Bitcoin Actions"]:::integ
        DSC["Discord Actions"]:::integ
    end

    %% Utility & DevOps Tier
    subgraph "Utility & DevOps"
        US["Utility Scripts & Functions"]:::util
        CI["CI/CD & Deployment"]:::util
    end

    %% Connections - Client Tier to Application Tier
    FE -->|"triggers"| RA
    FE -->|"renders"| RR
    RA -->|"API call"| NS

    %% Application Tier internal flow
    NS -->|"routes"| AR
    AR -->|"invokes"| BL

    %% Application Tier to Data Tier & Integration Tier
    BL -->|"writes/reads"| MY
    BL -->|"caches/queues"| RC
    BL -->|"integrates"| FI
    BL -->|"external call"| BTC
    BL -->|"external call"| DSC

    %% Utility interactions
    NS -->|"supports"| US
    CI -->|"deploys"| NS

    %% Class Definitions for colour coding
    classDef client fill:#f9e79f,stroke:#d4ac0d,stroke-width:2px;
    classDef apptier fill:#aed6f1,stroke:#3498db,stroke-width:2px;
    classDef datatier fill:#d5f5e3,stroke:#27ae60,stroke-width:2px;
    classDef integ fill:#f5cba7,stroke:#e67e22,stroke-width:2px;
    classDef util fill:#fadbd8,stroke:#e74c3c,stroke-width:2px;

    %% Click Events
    click FE "https://github.com/fabriclabs/sensemaker/tree/master/components"
    click RA "https://github.com/fabriclabs/sensemaker/tree/master/actions"
    click RR "https://github.com/fabriclabs/sensemaker/tree/master/reducers"
    click AS "https://github.com/fabriclabs/sensemaker/tree/master/assets"
    click NS "https://github.com/fabriclabs/sensemaker/blob/master/scripts/node.js"
    click AR "https://github.com/fabriclabs/sensemaker/tree/master/routes"
    click BL "https://github.com/fabriclabs/sensemaker/tree/master/services"
    click MY "https://github.com/fabriclabs/sensemaker/blob/master/knexfile.js"
    click RC "https://github.com/fabriclabs/sensemaker/blob/master/actions/redisActions.js"
    click FI "https://github.com/fabriclabs/sensemaker/blob/master/services/fabric.js"
    click BTC "https://github.com/fabriclabs/sensemaker/blob/master/actions/bitcoinActions.js"
    click DSC "https://github.com/fabriclabs/sensemaker/blob/master/actions/discordActions.js"
    click US "https://github.com/fabriclabs/sensemaker/tree/master/scripts"
    click CI "https://github.com/fabriclabs/sensemaker/tree/master/.github/workflows"
```
