# lumi-supercomputer/lumi-o-tools architecture

> Architecture diagram of the GitHub repository lumi-supercomputer/lumi-o-tools, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lumi-supercomputer/lumi-o-tools
- Repository: https://github.com/lumi-supercomputer/lumi-o-tools

## Overview

##### Project Type and Purpose
- The project is a command-line tool called "lumio-conf" whose main purpose is to automatically configure S3 authentication for several tools (rclone, s3cmd, awscli, boto3) used for accessing Lumi-O storage. The README explains that users can provide credentials which are then used to generate and update the configuration files for these S3-related tools.
- It is not a full-stack web application but rather a utility tool with a CLI interface that also interacts with external systems (e.g., a web authentication interface at https://auth.lumidata.eu/) and external tools (rclone, s3cmd, etc.).

##### File Structure Insights
- The top-level structure includes key files such as:
- Makefile and go.mod/go.sum for building the project.
- The “cmd” directory, which contains the main program (lumio-conf), representing the entry point of the application.
- The “internal” directory, which includes at least two sub-modules: “toolConfig” (handling specifics for different S3 tool configurations, such as aws_util.go, rclone_util.go, s3cmd_util.go, etc.) and “util” (general utilities).
- Additionally, the presence of shell scripts (e.g., curl_uppload.sh) indicates that there might be auxiliary tooling or deployment/testing scripts.

##### Key Components and Their Relationships
- CLI Entry Point (lumio-conf):
- Core executable (located in cmd/lumio-conf/main.go) that accepts user input, triggers authentication (via prompting the user and waiting for web-driven authentication at https://auth.lumidata.eu/), and uses environment variables to possibly override behavior.
- Configuration Modules (in internal/toolConfig):
- Each utility (rclone, s3cmd, awscli, boto3) has its dedicated configuration management module. These modules encapsulate methods to build and validate configuration files.
- Utility Functions (in internal/util):
- General helper functions used across the application.
- Legacy Build Configurations (oldEasybuild):
- Old configuration files for Easybuild and scripts for legacy deployment or tool version management.
- External Services and Tools:
- External web authentication interface where users fetch keys.
- External S3 tools such as rclone, s3cmd, awscli, boto3 that will use the generated configuration.
- Environment variables used to override or supply configuration information.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Internal Components
    A["CLI Application (lumio-conf)"]:::internal
    subgraph "Configuration Modules"
        B["rclone Configurator"]:::internal
        C["s3cmd Configurator"]:::internal
        D["awscli & boto3 Configurator"]:::internal
    end
    U["Utility Functions"]:::internal

    %% External Components
    E["External Web Authentication (auth.lumidata.eu)"]:::external
    subgraph "External S3 Tools"
        F["rclone Tool"]:::external
        G["s3cmd Tool"]:::external
        H["awscli Tool"]:::external
        I["boto3 Tool"]:::external
    end

    %% Legacy Component
    J["Legacy Build Configurations"]:::legacy

    %% Build & Dependency Management (each file as a node)
    subgraph "Build & Dependency Management"
        K["Makefile"]:::build
        L["go.mod"]:::build
        M["go.sum"]:::build
    end

    %% Auxiliary Script
    S["Auxiliary Shell Script (curl_uppload.sh)"]:::script

    %% Relationships from CLI Application to Configuration Modules
    A -->|"Generates_rclone_config"| B
    A -->|"Generates_s3cmd_config"| C
    A -->|"Generates_awscli/boto3_config"| D
    A -->|"Calls_utilities"| U
    A -->|"Triggers_authentication"| E

    %% Relationships from Configuration Modules to External S3 Tools
    B -->|"Deploys_config"| F
    C -->|"Deploys_config"| G
    D -->|"Deploys_config_to_awscli"| H
    D -->|"Deploys_config_to_boto3"| I

    %% Build & Dependency links to CLI Application
    K -->|"Build_info"| A
    L -->|"Build_info"| A
    M -->|"Build_info"| A

    %% Legacy and Auxiliary connections
    A -->|"Uses_legacy_scripts"| J
    A -->|"Executes_auxiliary_script"| S

    %% Click Events
    click A "https://github.com/lumi-supercomputer/lumi-o-tools/blob/main/cmd/lumio-conf/main.go"
    click B "https://github.com/lumi-supercomputer/lumi-o-tools/blob/main/internal/toolConfig/rclone_util.go"
    click C "https://github.com/lumi-supercomputer/lumi-o-tools/blob/main/internal/toolConfig/s3cmd_util.go"
    click D "https://github.com/lumi-supercomputer/lumi-o-tools/blob/main/internal/toolConfig/aws_util.go"
    click U "https://github.com/lumi-supercomputer/lumi-o-tools/blob/main/internal/util/util.go"
    click J "https://github.com/lumi-supercomputer/lumi-o-tools/tree/main/oldEasybuild"
    click K "https://github.com/lumi-supercomputer/lumi-o-tools/tree/main/Makefile"
    click L "https://github.com/lumi-supercomputer/lumi-o-tools/blob/main/go.mod"
    click M "https://github.com/lumi-supercomputer/lumi-o-tools/blob/main/go.sum"
    click S "https://github.com/lumi-supercomputer/lumi-o-tools/blob/main/curl_uppload.sh"

    %% Styles
    classDef internal fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef external fill:#FFE4B5,stroke:#000,stroke-width:2px;
    classDef legacy fill:#D3D3D3,stroke:#000,stroke-dasharray: 5,5;
    classDef build fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef script fill:#FFB6C1,stroke:#000,stroke-width:2px;
```
