# houbin/storagemanage architecture

> Architecture diagram of the GitHub repository houbin/storagemanage, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/houbin/storagemanage
- Repository: https://github.com/houbin/storagemanage

## Overview

##### Main Components
- Core Storage Management Service – This is the heart of the project and is implemented primarily in the “src” directory (for example, files like storage_api.cc, record_file.cc, record_reader.cc, and record_writer.cc). This component encapsulates the logic for interacting with storage devices and managing records, indexes, and file tables.
- gRPC Communication Layer – Both the “grpc” directory at the top level and the “doc/back/grpc” area indicate extensive use of gRPC (with supporting files like cJSON, watchdog, etc.). This layer likely provides external APIs and serves as the bridge for remote procedure calls for controlling or monitoring storage operations.
- Disk & Stream Management Modules – Components such as the stream_info, stream_op_handler, stream_transfer_client, and related “back” documentation files indicate specific handling of data streaming and partition management.
- Monitoring and Watchdog Service – With modules like watchdog_client (and associated grpc watchdog files), there is a dedicated monitoring subsystem that checks system health and stability.
- Installation & Shell Interfaces – The “install” and “doc/back/shell” directories include shell scripts (e.g., mount_one_disk.sh, format_one_disk.cc) for provisioning disk mounts, formatting operations, and boot setups, which support operational deployment.
- Testing & Utility Components – The “test” directory contains a range of unit tests and integration scripts (unit_test, test_thread, etc.) ensuring that each module functions as expected. The “util” directory is also available for miscellaneous helper functions.

##### Relationships and Interactions
- The Storage Management Service exposes an API layer (via storage_api.h and storage_api.cc) that is consumed by the store_client and store_client_center modules. These act as internal clients initiating storage operations.
- Disk management tasks (formatting, mounting, partition removal) are supported by both code modules within “src” and shell scripts in “install” and “doc/back/shell”. Represent these as auxiliary services or utilities that interact with the core system.
- There is a monitoring loop where the watchdog service constantly checks the health of storage operations—this can be shown as a feedback loop subscribing to system status.
- Libraries (found in “libs” for both arm and x86) should be noted as dependencies that provide low-level functionality such as JSON handling (via libcJSON) and gRPC operations.

##### Architectural Patterns & Design Principles
- Modular Monolith: Although not split into separate microservices, the project is modular, with distinct components (storage API, client, gRPC service, monitoring) that interact via well-defined interfaces.
- Clear Separation of Concerns: Each component (e.g., storage logic vs. communication vs. installation) has its separate directories and clearly delineated responsibilities.
- Extensibility: The presence of shell scripts and flexible configuration (e.g., config_opts headers, configuration files) hints that the system is designed to be extended and configured for various environments.
- Cross-Platform Support: The existence of separate makefiles and library builds for arm and x86 architectures indicates that the design must accommodate platform-specific differences.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Interaction Layer
    subgraph "Client Layer"
        Client("External Client"):::external
    end

    %% Communication Layer (gRPC)
    subgraph "Communication Layer"
        GRPC_dir("gRPC Directory"):::service
        GRPC_doc("Doc Back gRPC"):::service
    end

    %% Core Storage Management Service
    subgraph "Core Storage Management Service"
        API("Storage API (storage_api.cc)"):::core
        RecordFile("Record Management (record_file.cc)"):::core
        RecordReader("Record Reader (record_reader.cc)"):::core
        RecordWriter("Record Writer (record_writer.cc)"):::core
        IndexFile("Index Management (index_file.cc)"):::core
        FreeFileTable("Free File Table (free_file_table.cc)"):::core
    end

    %% Disk & Stream Management Modules
    subgraph "Disk & Stream Management Modules"
        StreamInfo("Stream Info (stream_info.cc)"):::module
        StreamOp("Stream Operation Handler (stream_op_handler.cc)"):::module
        StreamTransfer("Stream Transfer Client (stream_transfer_client.cc)"):::module
        StreamTransferManager("Stream Transfer Manager (stream_transfer_client_manager.cc)"):::module
        StreamIdManager("Stream ID Manager (stream_id_manager.cc)"):::module
    end

    %% Monitoring & Watchdog Service
    subgraph "Monitoring & Watchdog Service"
        WatchdogClient("Watchdog Client (src/watchdog_client.cc)"):::monitor
        GRPCWatchdog("gRPC Watchdog (grpc/watchdog.c)"):::monitor
    end

    %% Installation & Shell Interfaces
    subgraph "Installation & Shell Interfaces"
        InstallShell("Installation Shell Scripts"):::utility
        DocShell("Doc Back Shell Scripts"):::utility
    end

    %% Testing & Utility Components
    subgraph "Testing & Utility Components"
        Testing("Test Suite (test)"):::test
        Util("Utility Helpers (util)"):::test
    end

    %% Libraries & Dependencies
    subgraph "Libraries & Dependencies"
        Libs("External Libraries (cJSON, gRPC, tcmalloc)"):::library
    end

    %% Connections between Layers
    Client -->|"RPC_request"| GRPC_dir
    GRPC_dir -->|"internal"| GRPC_doc
    GRPC_doc -->|"calls"| API

    %% Connections within Core Storage Management Service
    API -->|"handles"| RecordFile
    API -->|"handles"| RecordReader
    API -->|"handles"| RecordWriter
    API -->|"handles"| IndexFile
    API -->|"handles"| FreeFileTable

    %% Interactions between Core and Disk/Stream Modules
    API -->|"initiates"| StreamInfo
    API -->|"initiates"| StreamOp
    StreamOp -->|"triggers"| StreamTransfer
    StreamTransfer -->|"manages"| StreamTransferManager
    StreamTransferManager -->|"coordinated_by"| StreamIdManager

    %% Monitoring interactions (feedback loop)
    API <-->|"monitored_by"| WatchdogClient
    WatchdogClient -->|"reports_to"| GRPCWatchdog

    %% Installation & Shell Interfaces support core operations
    InstallShell -->|"provisions"| API
    DocShell -->|"supports"| API

    %% Testing & Utility support Core functionality
    Testing -->|"validates"| API
    Util -->|"assists"| API

    %% Libraries support multiple layers
    Libs -->|"supports"| GRPC_dir
    Libs -->|"supports"| API

    %% Click Events for Core Storage Management Service
    click API "https://github.com/houbin/storagemanage/blob/master/src/storage_api.cc"
    click RecordFile "https://github.com/houbin/storagemanage/blob/master/src/record_file.cc"
    click RecordReader "https://github.com/houbin/storagemanage/blob/master/src/record_reader.cc"
    click RecordWriter "https://github.com/houbin/storagemanage/blob/master/src/record_writer.cc"
    click IndexFile "https://github.com/houbin/storagemanage/blob/master/src/index_file.cc"
    click FreeFileTable "https://github.com/houbin/storagemanage/blob/master/src/free_file_table.cc"

    %% Click Events for gRPC Communication Layer
    click GRPC_dir "https://github.com/houbin/storagemanage/tree/master/grpc/"
    click GRPC_doc "https://github.com/houbin/storagemanage/tree/master/doc/back/grpc/"

    %% Click Events for Disk & Stream Management Modules
    click StreamInfo "https://github.com/houbin/storagemanage/blob/master/doc/back/stream_info.cc"
    click StreamOp "https://github.com/houbin/storagemanage/blob/master/doc/back/stream_op_handler.cc"
    click StreamTransfer "https://github.com/houbin/storagemanage/blob/master/doc/back/stream_transfer_client.cc"
    click StreamTransferManager "https://github.com/houbin/storagemanage/blob/master/doc/back/stream_transfer_client_manager.cc"
    click StreamIdManager "https://github.com/houbin/storagemanage/blob/master/doc/back/stream_id_manager.cc"

    %% Click Events for Monitoring & Watchdog Service
    click WatchdogClient "https://github.com/houbin/storagemanage/blob/master/src/watchdog_client.cc"
    click GRPCWatchdog "https://github.com/houbin/storagemanage/blob/master/grpc/watchdog.c"

    %% Click Events for Installation & Shell Interfaces
    click InstallShell "https://github.com/houbin/storagemanage/blob/master/install/shell/boot_auto_mount.sh"
    click DocShell "https://github.com/houbin/storagemanage/blob/master/doc/back/shell/mount_one_disk.sh"

    %% Click Events for Testing & Utility Components
    click Testing "https://github.com/houbin/storagemanage/tree/master/test/"
    click Util "https://github.com/houbin/storagemanage/tree/master/util/"

    %% Styles
    classDef external fill:#f9c,stroke:#333,stroke-width:2px
    classDef service fill:#ccf,stroke:#333,stroke-width:2px
    classDef core fill:#cfc,stroke:#333,stroke-width:2px
    classDef module fill:#fcf,stroke:#333,stroke-width:2px
    classDef monitor fill:#ff9,stroke:#333,stroke-width:2px
    classDef utility fill:#cff,stroke:#333,stroke-width:2px
    classDef test fill:#fcc,stroke:#333,stroke-width:2px
    classDef library fill:#ccc,stroke:#333,stroke-width:2px
```
