# netskopeoss/netskope_log_analyzer architecture

> Architecture diagram of the GitHub repository netskopeoss/netskope_log_analyzer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/netskopeoss/netskope_log_analyzer
- Repository: https://github.com/netskopeoss/netskope_log_analyzer

## Overview

##### Project Type and Purpose
- This is a Splunk “app” (an add-on) called **Netskope Log Analyzer**.
- Purpose: Ingest Netskope log bundles, parse and index them into dedicated Splunk indexes, then provide prebuilt dashboards (XML views) and visualizations to help support teams interpret client-side, JSON, publisher, and CSV logs.

##### File Structure Analysis
- Top-level:
- LICENSE, README.md
- `local/` – Splunk app’s configuration directory:
– `app.conf`, `alert_actions.conf` – metadata about the app and any custom alert actions
– `inputs.conf` – defines file-monitoring inputs (/var/log/splunk/logs/*)
– `props.conf`, `transforms.conf` (if present) – field extractions and parsing rules
– `indexes.conf` – declares four dedicated indexes (netskope_client1, netskope_json1, etc.)
– `data/ui/` – XML definitions for dashboards and views
- `metadata/` – Splunk meta information (access controls)
- `static/` – static assets (e.g., dashboard screenshots)
- No custom code beyond Splunk’s XML and configs; logic lives in Splunk’s search and report definitions.

##### Key Insights from README
- Splunk forwarder/monitor watches `/var/log/splunk/logs` for Netskope log folder.
- Four dedicated indexes are created on the Splunk server.
- After installation and restart, users select a log folder in the dashboard UI to drive searches.

##### Main Components
A. **Data Source**
- Netskope log bundle (client logs, JSON logs, publisher logs, CSV logs)
B. **Splunk Universal Forwarder**
- Monitors `/var/log/splunk/logs/*`
- Securely forwards raw events to the Splunk Indexer
C. **Splunk Indexer**
- Reads `inputs.conf` routes events into four indexes:
- netskope_client1
- netskope_json1
- netskope_publisher1
- netskope_csv1
- Applies parsing rules from `props.conf` and `transforms.conf` (field extraction)
D. **Splunk Search Head & App**
- Holds the Netskope Log Analyzer app under `$SPLUNK_HOME/etc/apps/netskope_log_analyzer`
- Uses XML dashboards (`local/data/ui/views/*.xml`) to run saved searches/visualizations
- app.conf and alert_actions.conf define custom settings and any custom alerts
E. **User Interface**
- Splunk Web UI at `http://<splunk-server>:8000/en-GB/app/netskope_log_analyzer/`
- Dashboard tabs for configuration overview, connection info, steering, etc.

##### Architectural Patterns and Design Principles
- Separation of concerns: ingestion (forwarder), storage/indexing, search & visualization
- Configuration-driven parsing (props/transforms) rather than code
- Modular UI: each XML view focuses on a particular aspect of logs (e.g., client version)
- Extensibility: new views can be dropped into `local/data/ui/views`

##### Technologies, Frameworks, Libraries
- Splunk Universal Forwarder & Enterprise (Indexer + Search Head)
- Splunk’s configuration files (inputs.conf, props.conf, indexes.conf, alert_actions.conf)
- Dashboards written in Splunk’s XML dashboard format

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Source"
        Logs["Netskope Log Bundle"]
    end

    subgraph "Ingestion Layer"
        UF["Splunk Universal Forwarder"]:::compute
        Inputs["inputs.conf"]:::config
    end

    subgraph "Indexing Layer"
        Indexer["Splunk Indexer"]:::compute
        IndexCfg["indexes.conf"]:::config
        Props["props.conf"]:::config
    end

    subgraph "Storage Layer"
        ClientIdx["netskope_client1"]:::datastore
        JSONIdx["netskope_json1"]:::datastore
        PublisherIdx["netskope_publisher1"]:::datastore
        CSVIdx["netskope_csv1"]:::datastore
    end

    subgraph "Search Head & App"
        SH["Splunk Search Head"]:::compute
        AppMeta["app.conf"]:::config
        AlertCfg["alert_actions.conf"]:::config
        Nav["default.xml"]:::xml
        subgraph "Dashboard Views"
            V1["1_client-basic.xml"]:::xml
            V2["2_client-version.xml"]:::xml
            V3["3_client-connection.xml"]:::xml
            V4["4_client-steering.xml"]:::xml
            V5["5_client-npa-connection.xml"]:::xml
            V6["6_client-npa-conf.xml"]:::xml
            V7["iOS.xml"]:::xml
            V8["publisher.xml"]:::xml
            V9["update-check.xml"]:::xml
        end
    end

    subgraph "Presentation Layer"
        UI["Splunk Web UI"]:::ui
    end

    subgraph "App Assets"
        Meta["local.meta"]:::asset
        Static["static/"]:::asset
        Doc1["README.md"]:::asset
        Doc2["LICENSE"]:::asset
    end

    %% Connections
    Logs -->|monitored by| UF
    UF -->|forwards raw events| Indexer
    Indexer -->|parses using props.conf| Props
    Indexer -->|routes to indexes| ClientIdx
    Indexer --> ClientIdx
    Indexer --> JSONIdx
    Indexer --> PublisherIdx
    Indexer --> CSVIdx
    SH -->|runs searches on| ClientIdx
    SH --> ClientIdx
    SH --> JSONIdx
    SH --> PublisherIdx
    SH --> CSVIdx
    SH -->|serves dashboards| UI
    UI -->|user selects folder triggers searches| SH

    %% Click Events
    click Inputs "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/inputs.conf"
    click IndexCfg "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/indexes.conf"
    click Props "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/props.conf"
    click AppMeta "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/app.conf"
    click AlertCfg "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/alert_actions.conf"
    click Nav "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/nav/default.xml"
    click V1 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/1_client-basic.xml"
    click V2 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/2_client-version.xml"
    click V3 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/3_client-connection.xml"
    click V4 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/4_client-steering.xml"
    click V5 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/5_client-npa-connection.xml"
    click V6 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/6_client-npa-conf.xml"
    click V7 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/iOS.xml"
    click V8 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/publisher.xml"
    click V9 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/local/data/ui/views/update-check.xml"
    click Meta "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/metadata/local.meta"
    click Static "https://github.com/netskopeoss/netskope_log_analyzer/tree/main/static/"
    click Doc1 "https://github.com/netskopeoss/netskope_log_analyzer/blob/main/README.md"
    click Doc2 "https://github.com/netskopeoss/netskope_log_analyzer/tree/main/LICENSE"

    %% Styles
    classDef compute fill:#D6E9F8,stroke:#31708F,color:#31708F
    classDef datastore fill:#DFF0D8,stroke:#3C763D,color:#3C763D
    classDef config fill:#FCF8E3,stroke:#8A6D3B,color:#8A6D3B
    classDef xml fill:#F2DEDE,stroke:#A94442,color:#A94442
    classDef ui fill:#D9EDF7,stroke:#31708F,color:#31708F
    classDef asset fill:#EEEEEE,stroke:#888888,color:#888888
```
