# 1hehaq/shef architecture

> Architecture diagram of the GitHub repository 1hehaq/shef, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/1hehaq/shef
- Repository: https://github.com/1hehaq/shef

## Overview

**shef** is a **single-binary Go CLI tool** that lets a user query **Shodan “facet” aggregations** from the terminal **without an API key**, returning **pipe-friendly stdout** (optionally JSON). It’s effectively:

- **Terminal user → shef (local process) → Shodan website (public facet endpoint) → parsed facets → stdout**
- No persistent storage, no server component, no DB.
- “Integrates” with other tools primarily via **Unix pipelines** (httpx, amass, klik, yazi, etc.), but those are downstream consumers, not dependencies.

##### Runtime architecture (user ↔ binary ↔ Shodan)
2) **Build/Release architecture** (developer/CI ↔ GitHub Actions ↔ Go toolchain ↔ release artifacts)

##### User / Terminal
- Invokes `shef` with flags
- Consumes stdout (interactive or piped)

##### shef CLI binary (Go process)
- **Argument/Flag Parser**
- Handles `-q` (required), `-f` (default ip), `-list`, `-json`, `-h`
- **Facet Registry / Facet List Provider**
- Returns all supported facet types for `-list`
- Also supports shell autocomplete (`complete -W "$(shef -list)" shef`)
- **Query Builder**
- Encodes Shodan query string + facet parameter(s)
- Produces the final URL(s) to fetch
- **User-Agent Rotator**
- Selects a random UA per request (and possibly sets headers)
- **HTTP Client / Transport**
- Performs GET requests to Shodan
- Handles timeouts, redirects, non-200 responses, transient failures
- **Response Parser**
- Parses the returned content (likely HTML/embedded JSON; confirm via code/deps)
- Extracts the facet buckets/values (ip/domain/port/asn/vuln/etc.)
- **Output Renderer**
- “clean and pipe friendly output”
- Text mode: one value per line (likely)
- JSON mode: structured output to stdout (`-json`)
- **Error Handling / Exit Codes**
- Missing `-q`, no results, network errors, parse errors

##### External Service: Shodan Web
- The facet search endpoint(s) on `shodan.io`

- e.g., `httpx`, `amass`, `asnmap`, `klik`, `yazi`

##### Developer
- Pushes tags / commits

##### GitHub Actions (Release Workflow)
- Runs Go build
- Produces binaries / checksums / GitHub Release artifacts (depending on workflow)

##### End User installation
- `go install github.com/1hehaq/shef@latest` (Go module proxy + build on user machine)
- Optionally GitHub Releases download (if workflow publishes binaries)

##### Architectural patterns/design principles present (and how to show them)
- **CLI-first, Unix philosophy**
- “stdin/stdout oriented”, composable with pipes
- **Stateless execution**
- No DB, no caching (unless code shows otherwise)
- **Adapter pattern to external web service**
- shef acts as a thin adapter between “user query” and “Shodan facet UI responses”
- **Separation of concerns (logical)**

##### Labels
- “CLI args”
- “HTTP GET /search/facet?...”
- “HTML response (facet table)”
- “Extracted facet values”
- “stdout lines / JSON”
- Exact flags in the CLI boundary (`-q`, `-f`, `-list`, `-json`)

##### Directionality
- Request → Response
- Parse → Render
- shef → stdout

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =======================
  %% System Context (Runtime)
  %% =======================
  subgraph "Runtime (Local Execution)"
    direction TB

    User["Terminal User"]:::person

    subgraph "Local Machine"
      direction TB

      Shef["shef CLI (single Go binary)"]:::system

      subgraph "shef (Logical Components)"
        direction TB
        Flags["Argument/Flag Parser\n(-q required, -f default ip,\n-list, -json, -h)"]:::component
        FacetReg["Facet Registry / Facet List Provider\n(-list, autocomplete source)"]:::component
        QueryBuilder["Query Builder\n(construct Shodan facet URL + params)"]:::component
        UARotator["User-Agent Rotator\n(random UA + headers)"]:::component
        HTTPClient["HTTP Client / Transport\n(timeouts, redirects, status handling)"]:::component
        RespParser["Response Parser\n(parse HTML/embedded JSON → facet buckets)"]:::component
        Output["Output Renderer\n(text lines or JSON)"]:::component
        Err["Error Handling / Exit Codes\n(missing -q, no results, net/parse errors)"]:::component
      end

      Stdout["stdout\n(pipe-friendly text / JSON)"]:::artifact
      Stderr["stderr\n(errors)"]:::artifact
      Pipeline["Downstream CLI Tools\n(httpx, amass, yazi, ...)\n(consumers of stdout)"]:::externalOptional
    end

    Shodan["Shodan.io Web (Facet Search Endpoint)\n(no API key; web UI scraping/parse)"]:::external
  end

  %% =======================
  %% Build / Release Pipeline
  %% =======================
  subgraph "Build & Release (CI/CD)"
    direction TB

    Dev["Developer"]:::person
    Repo["GitHub Repo\n(source code)"]:::repo
    GHA["GitHub Actions\nrelease workflow"]:::ci
    GoTool["Go Toolchain\n(go build/test)"]:::build
    Artifacts["Release Artifacts\n(binaries/checksums)\n& Go Installable Module"]:::artifact
    EndUser["End User Installation\n(go install / download release)"]:::person
  end

  %% =======================
  %% Runtime Connections
  %% =======================
  User -->|"invokes shef with CLI args"| Shef
  Shef -->|"prints help (-h) / usage"| Stdout
  Shef -->|"errors + exit code"| Stderr

  Shef -->|"delegates"| Flags
  Flags -->|"if -list"| FacetReg
  FacetReg -->|"facet names list (offline)"| Stdout

  Flags -->|"if query mode (-q ...)"| QueryBuilder
  QueryBuilder -->|"build /search/facet?... query"| UARotator
  UARotator -->|"set headers (User-Agent)"| HTTPClient
  HTTPClient -->|"HTTP GET (facet search, no API key)"| Shodan
  Shodan -->|"HTTP response body (HTML/JSON-like)"| HTTPClient
  HTTPClient -->|"response bytes"| RespParser
  RespParser -->|"extracted facet values/buckets"| Output
  Output -->|"stdout lines or JSON (-json)"| Stdout
  Stdout -.->|"piped input"| Pipeline

  Flags -->|"invalid flags / missing -q"| Err
  HTTPClient -->|"non-200/timeout/redirect issues"| Err
  RespParser -->|"parse error / no results"| Err
  Err -->|"message + non-zero exit"| Stderr

  %% =======================
  %% Build/Release Connections
  %% =======================
  Dev -->|"push/commit/tag"| Repo
  Repo -->|"triggers workflow"| GHA
  GHA -->|"runs"| GoTool
  GoTool -->|"produces"| Artifacts
  Artifacts -->|"download binaries / go install"| EndUser

  %% =======================
  %% Click Events (Component Mapping)
  %% =======================
  click Shef "https://github.com/1hehaq/shef/blob/main/main.go"
  click Flags "https://github.com/1hehaq/shef/blob/main/main.go"
  click FacetReg "https://github.com/1hehaq/shef/blob/main/main.go"
  click QueryBuilder "https://github.com/1hehaq/shef/blob/main/main.go"
  click UARotator "https://github.com/1hehaq/shef/blob/main/main.go"
  click HTTPClient "https://github.com/1hehaq/shef/blob/main/main.go"
  click RespParser "https://github.com/1hehaq/shef/blob/main/main.go"
  click Output "https://github.com/1hehaq/shef/blob/main/main.go"
  click Err "https://github.com/1hehaq/shef/blob/main/main.go"

  click GHA "https://github.com/1hehaq/shef/blob/main/.github/workflows/release.yaml"
  click GoTool "https://github.com/1hehaq/shef/blob/main/go.mod"
  click Artifacts "https://github.com/1hehaq/shef/blob/main/go.sum"
  click Repo "https://github.com/1hehaq/shef/blob/main/README.md"

  %% =======================
  %% Styles
  %% =======================
  classDef person fill:#FFE8A3,stroke:#8A6D00,stroke-width:1.5px,color:#1f1f1f
  classDef system fill:#CFE8FF,stroke:#1D5EA8,stroke-width:1.8px,color:#0b2440
  classDef component fill:#E8F3FF,stroke:#3B82F6,stroke-width:1.2px,color:#0b2440
  classDef external fill:#FDE2E2,stroke:#B91C1C,stroke-width:1.5px,color:#3a0b0b
  classDef externalOptional fill:#FFF1F2,stroke:#FB7185,stroke-dasharray: 5 3,stroke-width:1.2px,color:#4a0b14
  classDef artifact fill:#E7F9ED,stroke:#15803D,stroke-width:1.3px,color:#062a14
  classDef ci fill:#EDE9FE,stroke:#6D28D9,stroke-width:1.6px,color:#1f1147
  classDef build fill:#E0F2FE,stroke:#0284C7,stroke-width:1.4px,color:#0a2533
  classDef repo fill:#F1F5F9,stroke:#334155,stroke-width:1.4px,color:#0f172a
```
