# arunx2/dashboard-sanitizer architecture

> Architecture diagram of the GitHub repository arunx2/dashboard-sanitizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arunx2/dashboard-sanitizer
- Repository: https://github.com/arunx2/dashboard-sanitizer

## Overview

##### Project Type and Purpose
- This is an open-source command-line utility (written in Go) that reads an Elastic/Kibana NDJSON file, “sanitizes” each dashboard object to be compatible with OpenSearch Dashboards, and writes out a cleaned NDJSON file.
- The tool is purely backend, no UI, no external DB—just file I/O and in-memory transformations.

##### File Structure
- Top‐level:
– main.go: orchestrates the application
– counter.go: likely tracks counts of objects processed/removed
– config/vars.go: defines CLI flags (source, output, version)
– model/model.go: defines Go structs matching the NDJSON object schema
– build.sh, GitHub Actions workflows: CI/CD pipeline
- No “frontend” or “database” folders; instead you have packages/modules that perform discrete tasks.

##### Extract Key Insights from the README
- The tool fixes version numbers and removes incompatible objects in NDJSON.
- Usage flags drive source and output file paths.
- It’s deprecated and merged into a larger project, but standalone architecture remains instructive.

##### Sketch the Main Components
a. CLI Interface
b. Config Loader (flags parsing)
c. File Reader (reads source NDJSON file line-by-line)
d. Parser/Unmarshaler (JSON → Go structs defined in model/)
e. Sanitizer Pipeline
– Version Fixer
– Incompatible Object Remover
– (Optional) Counter Module (tracks stats)
f. Serializer/Marshaler (Go structs → JSON)
g. File Writer (writes output NDJSON)
h. CI/CD Pipeline (GitHub Actions workflows, build.sh)

##### Relationships and Data Flows
- User invokes the CLI → CLI Interface
- CLI → Config Loader → populates parameters
- Config holds file paths → File Reader → streams raw JSON lines
- Each line → Parser → Go model object → Sanitizer → output object
- Sanitizer updates version fields, filters objects → passes to Serializer
- Serializer → File Writer → writes to output .ndjson
- Counter receives callbacks/events to tally processed/removed items
- CI/CD pipeline triggers on commit → runs build.sh & workflows → publishes artifacts

##### Architectural Patterns & Principles
- Modular/pipeline architecture: each stage is a separate package/module
- Single-responsibility: config, parsing, processing, output are decoupled
- In-memory streaming: efficient line-by-line processing of NDJSON
- Dependency management via Go modules (go.mod, go.sum)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input source file
    InputFile["Source NDJSON File"]:::io

    %% CLI & Config
    CLI["CLI Interface"]:::cli
    Config["Config Loader"]:::cli

    %% Core Processing
    Parser["Parser / Unmarshaler"]:::processing
    subgraph "Sanitizer Pipeline"
        direction TB
        VersionFixer["Version Fixer"]:::processing
        IncompatRemover["Incompatible Object Remover"]:::processing
        CounterMod["Counter Module"]:::processing
    end

    Serializer["Serializer / Marshaler"]:::processing

    %% I/O Output
    FileWriter["File Writer"]:::io
    OutputFile["Cleaned NDJSON File"]:::io

    %% CI/CD Pipeline
    subgraph "CI/CD Pipeline"
        direction TB
        BuildSh["build.sh"]:::cicd
        Workflow1["go.yml"]:::cicd
        Workflow2["go-ossf-slsa3-publish.yml"]:::cicd
        GoMod["go.mod"]:::cicd
        GoSum["go.sum"]:::cicd
    end

    %% Data Flow
    InputFile -->|reads| FileReader["File Reader"]:::io
    FileReader -->|line-by-line| Parser
    Parser -->|object| VersionFixer
    VersionFixer --> IncompatRemover
    IncompatRemover --> CounterMod
    CounterMod --> Serializer
    Serializer -->|writes| FileWriter
    FileWriter --> OutputFile

    %% CLI invocation flow
    CLI --> Config
    Config --> FileReader

    %% CI/CD triggers
    CLI -.->|commit triggers| BuildSh
    BuildSh --> Workflow1
    BuildSh --> Workflow2
    Workflow1 -->|uses| GoMod
    Workflow1 -->|uses| GoSum

    %% Click Events
    click CLI "https://github.com/arunx2/dashboard-sanitizer/blob/main/main.go"
    click Config "https://github.com/arunx2/dashboard-sanitizer/blob/main/config/vars.go"
    click FileReader "https://github.com/arunx2/dashboard-sanitizer/blob/main/main.go"
    click Parser "https://github.com/arunx2/dashboard-sanitizer/blob/main/model/model.go"
    click VersionFixer "https://github.com/arunx2/dashboard-sanitizer/blob/main/main.go"
    click IncompatRemover "https://github.com/arunx2/dashboard-sanitizer/blob/main/main.go"
    click CounterMod "https://github.com/arunx2/dashboard-sanitizer/blob/main/counter.go"
    click Serializer "https://github.com/arunx2/dashboard-sanitizer/blob/main/main.go"
    click FileWriter "https://github.com/arunx2/dashboard-sanitizer/blob/main/main.go"
    click BuildSh "https://github.com/arunx2/dashboard-sanitizer/blob/main/build.sh"
    click Workflow1 "https://github.com/arunx2/dashboard-sanitizer/blob/main/.github/workflows/go.yml"
    click Workflow2 "https://github.com/arunx2/dashboard-sanitizer/blob/main/.github/workflows/go-ossf-slsa3-publish.yml"
    click GoMod "https://github.com/arunx2/dashboard-sanitizer/blob/main/go.mod"
    click GoSum "https://github.com/arunx2/dashboard-sanitizer/blob/main/go.sum"

    %% Styles
    classDef cli fill:#E0F7FA,stroke:#0288D1,stroke-width:1px
    classDef processing fill:#E8F5E9,stroke:#388E3C,stroke-width:1px
    classDef io fill:#FFF3E0,stroke:#F57C00,stroke-width:1px
    classDef cicd fill:#ECEFF1,stroke:#757575,stroke-width:1px
```
