# splunk/splunk-sdk-python architecture

> Architecture diagram of the GitHub repository splunk/splunk-sdk-python, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/splunk/splunk-sdk-python
- Repository: https://github.com/splunk/splunk-sdk-python

## Overview

##### Project type and purpose
- This is an open-source SDK library (“Splunk Enterprise SDK for Python”) rather than a standalone application.
- Its purpose is to provide Python modules that developers can import into their own apps to connect to, index, search, and interact programmatically with a Splunk Enterprise instance via the Splunk REST API (and modular input and custom search command frameworks).

##### File structure
- Top-level artifacts:
– splunklib/          ← core SDK modules (client, binding, data, modularinput, searchcommands, utils)
– docs/               ← Sphinx documentation source and build scripts
– tests/              ← PyTest/unittest test suites (unit tests, integration tests, data fixtures)
– scripts/            ← helper scripts for environment setup and test orchestration
– docker-compose.yml  ← spins up a Splunk Enterprise container for integration tests
– Makefile            ← targets to build, test, and tear down local Docker Splunk instance
– setup.py, MANIFEST.in, .env, CI workflows
- Key patterns:
– Modular separation of SDK functionality into sub-packages
– Standard Python project layout with source, docs, tests, utilities
– CI integration (GitHub Actions, tox, pytest) and Docker-based integration testing

##### README for additional insights
- Confirms the Python versions supported, Splunk versions targeted, usage examples for connecting via username/password, tokens, session keys
- Highlights extensibility points: custom search commands (StreamingCommand, GeneratingCommand), modular inputs (InputDefinition), and the service object for API calls
- Describes developer workflows: adding fields, generating records, setting up logging

a. Developer Application Layer
– Your Python application imports splunklib.client, splunklib.searchcommands, etc.
– It calls client.connect(...) to obtain a Service object.
b. Splunk SDK Core (splunklib)
– binding.py: HTTP/REST call construction & response parsing
– client.py: high-level Service and namespace abstractions
– data.py, results.py: wrappers for indexed/search results
– modularinput/: classes for building custom modular inputs and streaming events
– searchcommands/: base classes & decorators for custom search commands (streaming, generating, reporting, eventing)
– utils.py: shared helpers
c. External Splunk Enterprise Service
– Exposes REST endpoints on port 8089 (configured via .env/docker-compose)
– The SDK’s REST calls flow over HTTPS/TCP to this service
d. Test & CI Infrastructure
– docker-compose.yml spins up a Splunk container
– pytest + Makefile orchestrate integration tests against the running container
– GitHub Actions workflows invoke make test, lint, release

##### Architectural patterns & design principles
- Modular package structure for single-responsibility (client, binding, commands, inputs)
- Adapter/wrapper pattern around Splunk’s REST API
- Extension points via abstract base classes (StreamingCommand, GeneratingCommand, Script)
- Separation between core library and examples/tests

##### Elements
- Clear labels for each component: Developer App, each splunklib module, Splunk Enterprise Service, Test Harness
- Developer App → splunklib.client → splunklib.binding → Splunk REST API
- Developer App → splunklib.searchcommands → Splunk command framework
- CI Pipeline → Docker Compose → Splunk Container ← Test Runner
- External systems (Splunk Enterprise container) in another (e.g., green)
- CI/Test components in a third (e.g., orange)

##### Detailed capture of essential elements
- Don’t omit the modularinput and searchcommands sub-packages—they are key extension points
- How the .env file and docker-compose.yml configure host, port, credentials for local tests
- Sphinx docs pipeline (docs/ → HTML) as a peripheral component
- CI workflows (GitHub Actions → pytest/tox → coverage reports)

## Diagram (Mermaid)

```mermaid
graph TB
    %% Developer Application Layer
    DeveloperApp["Developer Application"]:::app

    %% Splunk SDK Core
    subgraph "Splunk SDK Core"
        direction TB
        Client["splunklib.client"]:::library
        Binding["splunklib.binding"]:::library
        Data["splunklib.data & splunklib.results"]:::library
        Utils["splunklib.utils"]:::library
        ModularInput["splunklib.modularinput"]:::library
        SearchCommands["splunklib.searchcommands"]:::library
    end

    %% External Splunk Service
    SplunkService["Splunk Enterprise Service\n(REST API)"]:::external

    %% CI & Test Infrastructure
    subgraph "CI & Test Infrastructure"
        direction TB
        CI["CI Pipeline\n(GitHub Actions)"]:::ci
        MakefileNode["Makefile & scripts"]:::ci
        DockerCompose["docker-compose.yml & .env"]:::ci
        TestRunner["pytest / tox"]:::ci
        SplunkContainer["Splunk Container"]:::external
        Docs["Sphinx Docs (docs/)"]:::docs
    end

    %% Connections
    DeveloperApp -->|"imports"| Client
    Client -->|"uses"| Binding
    Binding -->|"REST calls"| SplunkService
    DeveloperApp -->|"uses extension"| SearchCommands
    DeveloperApp -->|"uses extension"| ModularInput
    Client -->|"wraps data"| Data
    Client -->|"helpers"| Utils

    CI -->|triggers| MakefileNode
    MakefileNode -->|orchestrates| DockerCompose
    DockerCompose -->|spins up| SplunkContainer
    TestRunner -->|runs tests against| SplunkContainer
    CI -->|runs| TestRunner
    Docs -->|builds| HTML

    %% Click Events for component mapping
    click Client "https://github.com/splunk/splunk-sdk-python/blob/master/splunklib/client.py"
    click Binding "https://github.com/splunk/splunk-sdk-python/blob/master/splunklib/binding.py"
    click Data "https://github.com/splunk/splunk-sdk-python/blob/master/splunklib/data.py"
    click Data "https://github.com/splunk/splunk-sdk-python/blob/master/splunklib/results.py"
    click Utils "https://github.com/splunk/splunk-sdk-python/blob/master/splunklib/utils.py"
    click ModularInput "https://github.com/splunk/splunk-sdk-python/blob/master/splunklib/modularinput/__init__.py"
    click SearchCommands "https://github.com/splunk/splunk-sdk-python/blob/master/splunklib/searchcommands/__init__.py"
    click DockerCompose "https://github.com/splunk/splunk-sdk-python/blob/master/docker-compose.yml"
    click MakefileNode "https://github.com/splunk/splunk-sdk-python/tree/master/Makefile"
    click Docs "https://github.com/splunk/splunk-sdk-python/tree/master/docs/"

    %% Styles
    classDef library fill:#D0E6FF,stroke:#0366D6,stroke-width:1px
    classDef external fill:#DFFFE0,stroke:#228B22,stroke-width:1px
    classDef ci fill:#FFE4B5,stroke:#FF8C00,stroke-width:1px
    classDef app fill:#F0F0F0,stroke:#666666,stroke-width:1px
    classDef docs fill:#F5F5F5,stroke:#999999,stroke-width:1px
```
