# flashcatcloud/categraf architecture

> Architecture diagram of the GitHub repository flashcatcloud/categraf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/flashcatcloud/categraf
- Repository: https://github.com/flashcatcloud/categraf

## Overview

##### Main Components

a. Core Components:
- Input Plugins System (located in /inputs)
- Writer System (located in /writer)
- Agent System (located in /agent)
- Logs System (located in /logs)
- Parser System (located in /parser)

b. Protocol Support:
- Prometheus Remote Write
- Falcon Protocol
- OpenTSDB Protocol
- Pushgateway Protocol

c. Collection Targets:
- Physical Machines
- Virtual Machines
- Containers/Kubernetes
- Network Devices
- Databases/Middleware
- Logs

##### Architectural Layers

Layer 1 - Data Collection:
- Input plugins connecting to various data sources
- Plugin manager system
- Log collection pipeline

Layer 2 - Processing:
- Parser subsystem for different formats
- Metric processing and aggregation
- Log processing pipeline

Layer 3 - Output:
- Writer system with multiple protocol support
- Remote write capabilities
- Buffer and batch processing

##### Key Design Elements

- Rectangles for core components
- Hexagons for plugins

- Blue: Core components
- Green: Input plugins
- Orange: Processing components
- Purple: Output writers
- Gray: External systems

##### Implementation Details

Create sections showing:
- Plugin architecture with the provider manager
- Log pipeline flow
- Metric processing pipeline
- Configuration management
- Health monitoring system

##### Interactions

Data flow between:
- Input plugins → Parser → Processor → Writer
- Log collectors → Log pipeline → Output destinations
- Agent communication with remote systems
- Configuration management flow

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Collection
        direction TB
        InputPlugins["Input Plugins System"]:::core
        PluginManager["Plugin Manager"]:::core
        PluginCollection["Plugin Collection"]:::core
        InputPlugins --> PluginManager
        InputPlugins --> PluginCollection
    end

    subgraph Processing
        direction TB
        Parser["Parser System"]:::core
        PrometheusParser["Prometheus Parser"]:::parser
        FalconParser["Falcon Parser"]:::parser
        Parser --> PrometheusParser
        Parser --> FalconParser
    end

    subgraph AgentSystem
        direction TB
        CoreAgent["Core Agent"]:::core
        MetricsAgent["Metrics Agent"]:::core
        LogsAgent["Logs Agent"]:::core
        CoreAgent --> MetricsAgent
        CoreAgent --> LogsAgent
    end

    subgraph LogsSystem
        direction TB
        LogsPipeline["Logs Pipeline"]:::core
        LogsParser["Logs Parser"]:::core
        LogsClient["Logs Client"]:::core
        LogsPipeline --> LogsParser
        LogsPipeline --> LogsClient
    end

    subgraph Writers
        direction TB
        CoreWriter["Core Writer"]:::core
        WritersManager["Writers Manager"]:::core
        CoreWriter --> WritersManager
    end

    subgraph Protocols
        direction TB
        PrometheusRW["Prometheus Remote Write"]:::protocol
        FalconProto["Falcon Protocol"]:::protocol
        OpenTSDBProto["OpenTSDB Protocol"]:::protocol
        PushgatewayProto["Pushgateway Protocol"]:::protocol
    end

    subgraph Config
        direction TB
        CoreConfig["Core Config"]:::config
        LogsConfig["Logs Config"]:::config
    end

    subgraph Metrics
        direction TB
        CoreMetrics["Core Metrics"]:::metrics
        SampleProcessing["Sample Processing"]:::metrics
    end

    Collection --> Processing
    Processing --> Writers
    AgentSystem --> Writers
    LogsSystem --> Writers
    Writers --> Protocols
    Config --> Collection
    Config --> LogsSystem
    Metrics --> Processing

    classDef core fill:#3498db,stroke:#2980b9,color:white
    classDef parser fill:#e67e22,stroke:#d35400,color:white
    classDef protocol fill:#9b59b6,stroke:#8e44ad,color:white
    classDef config fill:#2ecc71,stroke:#27ae60,color:white
    classDef metrics fill:#e74c3c,stroke:#c0392b,color:white

    click InputPlugins "https://github.com/flashcatcloud/categraf/tree/main//inputs"
    click PluginManager "https://github.com/flashcatcloud/categraf/blob/main//inputs/provider_manager.go"
    click PluginCollection "https://github.com/flashcatcloud/categraf/blob/main//inputs/collector.go"
    click CoreWriter "https://github.com/flashcatcloud/categraf/blob/main//writer/writer.go"
    click WritersManager "https://github.com/flashcatcloud/categraf/blob/main//writer/writers.go"
    click CoreAgent "https://github.com/flashcatcloud/categraf/blob/main//agent/agent.go"
    click MetricsAgent "https://github.com/flashcatcloud/categraf/blob/main//agent/metrics_agent.go"
    click LogsAgent "https://github.com/flashcatcloud/categraf/blob/main//agent/logs_agent.go"
    click LogsPipeline "https://github.com/flashcatcloud/categraf/blob/main//logs/pipeline/pipeline.go"
    click LogsParser "https://github.com/flashcatcloud/categraf/blob/main//logs/parser/parser.go"
    click LogsClient "https://github.com/flashcatcloud/categraf/blob/main//logs/client/destinations.go"
    click Parser "https://github.com/flashcatcloud/categraf/blob/main//parser/parser.go"
    click PrometheusParser "https://github.com/flashcatcloud/categraf/blob/main//parser/prometheus/parser.go"
    click FalconParser "https://github.com/flashcatcloud/categraf/blob/main//parser/falcon/parser.go"
    click PrometheusRW "https://github.com/flashcatcloud/categraf/blob/main//api/router_remotewrite.go"
    click FalconProto "https://github.com/flashcatcloud/categraf/blob/main//api/router_falcon.go"
    click OpenTSDBProto "https://github.com/flashcatcloud/categraf/blob/main//api/router_opentsdb.go"
    click PushgatewayProto "https://github.com/flashcatcloud/categraf/blob/main//api/router_pushgateway.go"
    click CoreConfig "https://github.com/flashcatcloud/categraf/blob/main//config/config.go"
    click LogsConfig "https://github.com/flashcatcloud/categraf/blob/main//config/logs/config.go"
    click CoreMetrics "https://github.com/flashcatcloud/categraf/blob/main//types/metric/metric.go"
    click SampleProcessing "https://github.com/flashcatcloud/categraf/blob/main//types/sample.go"
```
