# comcast/fishymetrics architecture

> Architecture diagram of the GitHub repository comcast/fishymetrics, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/comcast/fishymetrics
- Repository: https://github.com/comcast/fishymetrics

## Overview

##### Project Type and Purpose
- This is a backend service – a Prometheus exporter named “fishymetrics.”
- It scrapes hardware management stats from bare-metal chassis via the Redfish API (and pluggable non-Redfish APIs) and exposes metrics over HTTP for Prometheus.
- It also provides build-info and runtime metrics endpoints, supports credential management via Vault or static config, and offers pluggable extensions for additional hardware models.

##### File Structure Analysis
Top-level layout:
- cmd/fishymetrics – the main CLI/app entry point and template handling
- exporter – HTTP handlers, core scraper logic, metrics construction, plus a nested moonshot exporter
- oem – model-specific Redfish wrappers per chassis type (generic + moonshot folder)
- plugins – extension points for non-Redfish or custom collectors (e.g., Nuova)
- common, config, pool, vault, logger, middleware – cross-cutting libraries for credentials, configuration, concurrency, secret retrieval, logging, and HTTP middleware
- helm – Kubernetes deployment charts
- Dockerfile/Makefile – build/test/deploy automation
- .github – CI/security workflows and issue templates

##### Key Architectural Insights from README
- Single HTTP service exposing /metrics, /scrape, /info endpoints
- Flags/env-var configuration for BMC creds, Vault integration, logging, collector filters
- Prometheus scrapes the /scrape path with URL params to target specific chassis models/addresses
- Concurrency handled via a worker pool for parallel Redfish calls
- Middleware for request logging and Prometheus internal metrics

##### Main Components
a. fishymetrics Server
– cmd/fishymetrics: application bootstrap, flag parsing, HTTP server start
b. HTTP Layer
– middleware (logging, muxprom)
– handlers (/metrics, /info, /scrape) in exporter/handlers.go
c. Credential Management
– static credentials or Vault integration (vault package + common/credentials)
d. Scraping Layer
– exporter core: orchestrates collectors, applies exclude filters
– pool: concurrency management for parallel Redfish/tasks
– oem modules: Redfish API clients per chassis model
– plugins: custom handlers for non-Redfish targets
e. Metrics Assembly
– exporter/metrics.go: builds Prometheus metric families
– Prometheus client library
f. External Dependencies
– Redfish-enabled BMCs (HTTP REST endpoints)
– Vault server (for AppRole credential retrieval)
– Prometheus (scrapes metrics)
– Vector or file sink (for structured logging)

##### Relationships and Data Flow
1. Prometheus → HTTP GET /scrape on fishymetrics (with query params)
2. HTTP middleware logs request + records internal metrics
3. Handler invokes credential loader (Vault or static)
4. Handler submits tasks to pool: for each collector (oem module or plugin)
5. Collector contacts BMC Redfish API (HTTPS) and returns raw JSON/stats
6. Aggregated data passed to exporter/metrics, converted into Prometheus metrics
7. /metrics endpoint serves exporter’s own runtime metrics
8. Logs emitted to stdout/file or Vector endpoint

##### Important Architectural Patterns and Principles
- Modular design: separate packages for concerns (config, logging, HTTP, scraping)
- Pluggable collectors via plugins/ and exporter/moonshot
- Concurrency via a worker pool to bound parallel Redfish calls
- Middleware for cross-cutting concerns (logging, instrumentation)
- Twelve-factor style config (flags + env vars)
- Containerized deployment with Docker + Helm

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    Prometheus["Prometheus"]:::external
    BMC["BMC Redfish API"]:::external
    VaultSvc["Vault Server"]:::external
    LogSink["Logging Sink\n(vector/file)"]:::external

    %% Application Entry
    subgraph "fishymetrics Server"
        Bootstrap["Bootstrap\nmain.go"]:::service
        Templates["Templates\ntemplates.go"]:::service
    end
    click Bootstrap "https://github.com/comcast/fishymetrics/blob/main/cmd/fishymetrics/main.go"
    click Templates "https://github.com/comcast/fishymetrics/blob/main/cmd/fishymetrics/templates.go"

    %% HTTP and Middleware Layer
    subgraph "HTTP Layer"
        Handler["HTTP Handlers\nexporter/handlers.go"]:::service
    end
    click Handler "https://github.com/comcast/fishymetrics/blob/main/exporter/handlers.go"

    subgraph "Middleware"
        LogMW["Logging Middleware"]:::library
        PromMW["Instrumentation Middleware"]:::library
    end
    click LogMW "https://github.com/comcast/fishymetrics/blob/main/middleware/logging/logging.go"
    click PromMW "https://github.com/comcast/fishymetrics/blob/main/middleware/muxprom/middleware.go"

    %% Credential Management
    Credential["Credential Loader\n(common & vault)"]:::service
    click Credential "https://github.com/comcast/fishymetrics/blob/main/common/credentials.go"
    click Credential "https://github.com/comcast/fishymetrics/blob/main/vault/vault.go"

    %% Scraping Orchestration
    subgraph "Scraping Orchestration"
        Orchestrator["Exporter Core\nexporter/exporter.go"]:::service
        Helpers["Helpers\nexporter/helpers.go"]:::library
    end
    click Orchestrator "https://github.com/comcast/fishymetrics/blob/main/exporter/exporter.go"
    click Helpers "https://github.com/comcast/fishymetrics/blob/main/exporter/helpers.go"

    %% Worker Pool
    Pool["Worker Pool\npool/pool.go"]:::library
    Task["Task Definition\npool/task.go"]:::library
    click Pool "https://github.com/comcast/fishymetrics/blob/main/pool/pool.go"
    click Task "https://github.com/comcast/fishymetrics/blob/main/pool/task.go"

    %% OEM Collectors
    subgraph "OEM Collectors"
        OEMGeneric["Generic Collectors\n(oem/*.go)"]:::service
        OEMMoon["Moonshot Collectors\n(oem/moonshot/*.go)"]:::service
    end
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/chassis.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/drive.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/firmware.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/manager.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/memory.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/network_adapter.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/power.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/processor.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/storage_ctrl.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/system.go"
    click OEMGeneric "https://github.com/comcast/fishymetrics/blob/main/oem/thermal.go"
    click OEMMoon "https://github.com/comcast/fishymetrics/tree/main/oem/moonshot"

    %% Plugin Collectors
    Plugin["Plugin Collectors\nplugins/nuova"]:::service
    click Plugin "https://github.com/comcast/fishymetrics/blob/main/plugins/nuova/nuova.go"

    %% Metrics Assembly
    Metrics["Metrics Assembly\nexporter/metrics.go"]:::service
    MoonMetrics["Moonshot Metrics\nexporter/moonshot/metrics.go"]:::service
    click Metrics "https://github.com/comcast/fishymetrics/blob/main/exporter/metrics.go"
    click MoonMetrics "https://github.com/comcast/fishymetrics/blob/main/exporter/moonshot/metrics.go"

    %% Data Flows
    Prometheus -->|"GET /scrape"| LogMW
    LogMW --> PromMW
    PromMW --> Handler
    Handler -->|"load credentials"| Credential
    Handler --> VaultSvc
    Credential --> Orchestrator
    Orchestrator --> Helpers
    Orchestrator --> Pool
    Pool --> Task
    Task --> OEMGeneric
    Task --> OEMMoon
    Task --> Plugin
    OEMGeneric -->|"HTTPS"| BMC
    OEMMoon -->|"HTTPS"| BMC
    Plugin -->|"Custom API"| BMC
    OEMGeneric --> Metrics
    OEMMoon --> Metrics
    Plugin --> Metrics
    Metrics -->|"expose /metrics"| Prometheus
    Handler --> LogSink

    %% Styles
    classDef service fill:#d0e7ff,stroke:#0267c1
    classDef library fill:#f9f7a1,stroke:#c4a000
    classDef external fill:#dfffe0,stroke:#2d862d
```
