How ibmstorage/ibm-storageinsights-ecosystem-prometheus works
This project is an exporter written in Go that bridges IBM Storage Insights metrics into Prometheus, acting as a Prometheus endpoint that gathers metrics from the IBM Storage Insights REST APIs. The exporter is organized as multiple Go modules (config, simetrics, main), each dedicated to a specific functional area.
Main Components
• Prometheus Server: An external monitoring system that scrapes metrics from this exporter.
• IBM Storage Insights REST API: The external API endpoints (Token API and Metrics API) from which the exporter fetches metrics.
• Exporter Server Process: Consists of three submodules:
• The “main” module that starts the server, handles incoming scrape requests (via HTTP endpoints on a defined port such as 8089), and orchestrates metric collection.
• The “config” module which loads and validates configuration (such as the API key, tenant UUID, and metric types) from a config.json file.
• The “simetrics” module that houses the core logic (in si-metrics.go) for calling the Storage Insights REST APIs, handling token authentication, and translating the fetched metrics into Prometheus format.
Relationships and Interactions
• When Prometheus initiates a scrape (HTTP GET) on the exporter endpoint—the request is received by the main module.
• The main module uses the configuration loaded (via the config module) to know what metrics to fetch and authenticate details.
• It then delegates the actual API call and metric transformation to the simetrics module. Here, the exporter first ensures a valid authentication token (by calling the Token API if needed) and then synchronously calls the Metrics API.
• The simetrics module converts the raw data from the Storage Insights APIs into the Prometheus exposition format, which is then returned to Prometheus as the scrape response.
Architectural Patterns and Design Principles
• Separation of Concerns: The system is modularized into separate Go modules (config, simetrics, main), each handling a different aspect (configuration management, metrics API interaction, and server orchestration, respectively).
• Synchronous Data Flow on Scrape: Every time Prometheus scrapes the endpoint, there is a synchronous chain of operations (token fetch if necessary, API call, data transformation) ensuring up-to-date metrics.
• Resilience in API Authentication: The exporter manages API token validity and renews it automatically every 15 minutes, which is a critical design element worth noting.
Technologies and Frameworks
• Go Language: The project is implemented in Go.
• Prometheus: The exporter conforms to Prometheus’s metric format and is designed to be scraped by Prometheus.
• REST APIs: The external IBM Storage Insights REST APIs are central to fetching the performance and capacity metrics.