How arunx2/dashboard-sanitizer works
Project Type and Purpose
• This is an open-source command-line utility (written in Go) that reads an Elastic/Kibana NDJSON file, “sanitizes” each dashboard object to be compatible with OpenSearch Dashboards, and writes out a cleaned NDJSON file.
• The tool is purely backend, no UI, no external DB—just file I/O and in-memory transformations.
File Structure
• Top‐level:
– main.go: orchestrates the application
– counter.go: likely tracks counts of objects processed/removed
– config/vars.go: defines CLI flags (source, output, version)
– model/model.go: defines Go structs matching the NDJSON object schema
– build.sh, GitHub Actions workflows: CI/CD pipeline
• No “frontend” or “database” folders; instead you have packages/modules that perform discrete tasks.
Extract Key Insights from the README
• The tool fixes version numbers and removes incompatible objects in NDJSON.
• Usage flags drive source and output file paths.
• It’s deprecated and merged into a larger project, but standalone architecture remains instructive.
Sketch the Main Components
a. CLI Interface
b. Config Loader (flags parsing)
c. File Reader (reads source NDJSON file line-by-line)
d. Parser/Unmarshaler (JSON → Go structs defined in model/)
e. Sanitizer Pipeline
– Version Fixer
– Incompatible Object Remover
– (Optional) Counter Module (tracks stats)
f. Serializer/Marshaler (Go structs → JSON)
g. File Writer (writes output NDJSON)
h. CI/CD Pipeline (GitHub Actions workflows, build.sh)
Relationships and Data Flows
• User invokes the CLI → CLI Interface
• CLI → Config Loader → populates parameters
• Config holds file paths → File Reader → streams raw JSON lines
• Each line → Parser → Go model object → Sanitizer → output object
• Sanitizer updates version fields, filters objects → passes to Serializer
• Serializer → File Writer → writes to output .ndjson
• Counter receives callbacks/events to tally processed/removed items
• CI/CD pipeline triggers on commit → runs build.sh & workflows → publishes artifacts
Architectural Patterns & Principles
• Modular/pipeline architecture: each stage is a separate package/module
• Single-responsibility: config, parsing, processing, output are decoupled
• In-memory streaming: efficient line-by-line processing of NDJSON
• Dependency management via Go modules (go.mod, go.sum)