# hik8s/core architecture

> Architecture diagram of the GitHub repository hik8s/core, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hik8s/core
- Repository: https://github.com/hik8s/core

## Overview

##### Project type and purpose
- This is a microservices‐style data‐processing and backend system for a Kubernetes chatbot (“HiK8s Core”), implemented in Rust.
- It consists of multiple standalone services (each with its own Cargo.toml and main.rs), plus a shared library and end-to-end tests.

##### File structure
- Top‐level “rs/” directory holds subprojects:
- data-intake (HTTP ingest service)
- data-processing (event processing workers)
- data-vectorizer (embeddings/vectorization)
- chat-backend (chat API server)
- shared (common code: DB connectors, message-broker clients, OpenAI client, routing, types)
- tests (E2E integration tests)
- CI/CD workflows and Dockerfile at root.
- Config files for Rocket (chat-backend), Clippy, Buck2, plus Docker and README.

##### README for additional insights
- External dependencies: GreptimeDB (time-series), Qdrant (vector DB), Redis (cache), Fluvio (message broker), OpenAI.
- Data flows: ingest → processing → vectorize → store/query → chat API.

##### Detail
- Microservice as its own component
- Intermediate Fluvio topics if desired (e.g., “raw-logs”, “processed-events”)
- Authentication/authorization if part of shared router (e.g., JWT guard)
- Retry or error-handling paths if notable

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entities
    Clients["Clients"]:::external
    Shared["Shared Library\n(Common Code)"]:::shared

    %% Microservices
    subgraph "Microservices"
        direction TB
        Intake["Data Intake Service\n(Rocket/Rust)"]:::service
        Proc["Data Processing Service\n(Rust)"]:::service
        Vector["Data Vectorizer Service\n(Rust)"]:::service
        Chat["Chat Backend Service\n(Rocket/Rust)"]:::service
    end

    %% Message Broker
    Fluvio["Fluvio Broker"]:::broker

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        GT["GreptimeDB"]:::db
        QR["Qdrant"]:::db
        RD["Redis Cache"]:::db
    end

    %% External API
    OpenAI["OpenAI API"]:::external

    %% Test Suite
    Tests["E2E Tests"]:::test

    %% Data Flows
    Clients -->|"HTTP POST raw logs"| Intake
    Intake -->|"produce raw-logs"| Fluvio
    Fluvio -->|"consume raw-logs"| Proc
    Proc -->|"produce processed-events"| Fluvio
    Fluvio -->|"consume processed-events"| Vector
    Vector -->|"write embeddings"| QR
    Vector -->|"write metrics"| GT
    Chat -->|"vector similarity queries"| QR
    Chat -->|"time-series queries"| GT
    Chat -->|"session/cache access"| RD
    Chat -->|"chat completions"| OpenAI

    %% Shared dependencies
    Shared --> Intake
    Shared --> Proc
    Shared --> Vector
    Shared --> Chat

    %% Test integration
    Tests --> Intake
    Tests --> Chat

    %% Click Events
    click Intake "https://github.com/hik8s/core/tree/main/rs/data-intake/"
    click Proc "https://github.com/hik8s/core/tree/main/rs/data-processing/"
    click Vector "https://github.com/hik8s/core/tree/main/rs/data-vectorizer/"
    click Chat "https://github.com/hik8s/core/tree/main/rs/chat-backend/"
    click Fluvio "https://github.com/hik8s/core/tree/main/rs/shared/src/connections/fluvio/"
    click GT "https://github.com/hik8s/core/tree/main/rs/shared/src/connections/greptime/"
    click QR "https://github.com/hik8s/core/tree/main/rs/shared/src/connections/qdrant/"
    click RD "https://github.com/hik8s/core/tree/main/rs/shared/src/connections/redis/"
    click OpenAI "https://github.com/hik8s/core/tree/main/rs/shared/src/connections/openai/"
    click Shared "https://github.com/hik8s/core/tree/main/rs/shared/"
    click Tests "https://github.com/hik8s/core/tree/main/rs/tests/"

    %% Styles
    classDef service fill:#cfe2f3,stroke:#6fa8dc,stroke-width:2px;
    classDef broker fill:#f9cb9c,stroke:#e69138,stroke-width:2px;
    classDef db fill:#d9ead3,stroke:#6aa84f,stroke-width:2px;
    classDef external fill:#d9d2e9,stroke:#a64d79,stroke-width:2px;
    classDef shared fill:#f2f2f2,stroke:#999999,stroke-width:2px,stroke-dasharray: 5 5;
    classDef test fill:#ffe599,stroke:#b45f06,stroke-width:2px;
```
