# realkaiviljoen/streamx architecture

> Architecture diagram of the GitHub repository realkaiviljoen/streamx, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/realkaiviljoen/streamx
- Repository: https://github.com/realkaiviljoen/streamx

## Overview

streamx is a command-line tool that probes a remote archive, then streams it directly into extracted files without staging the compressed archive. Its main workflow selects a RAR5 checkpointing engine or a general streaming engine, reconnects at verified byte offsets, and persists progress for resume. The RAR5 path parses archive blocks and can checkpoint stored entries; other formats are network-resumable but not crash-resumable. The README documents major behavior beyond the sampled excerpts; unsampled implementation details are represented only where paths are present in the tree.

Main entry points: [`cmd/streamx/cmds.go`](https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/cmds.go) (Probe and run), [`internal/extract/rar5.go`](https://github.com/realkaiviljoen/streamx/blob/main/internal/extract/rar5.go) (RAR5 extraction), [`cmd/streamx/main.go`](https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/main.go) (Command router), [`internal/httpx/httpx.go`](https://github.com/realkaiviljoen/streamx/blob/main/internal/httpx/httpx.go) (HTTP probe)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_commands["CLI commands"]
  node_cli["Command router<br/>[main.go]"]
  node_commands_impl["Probe and run<br/>[cmds.go]"]
  node_status["Status inspection<br/>[cmds.go]"]
  node_progress["Progress display<br/>[progress.go]"]
end

subgraph group_transfer["Remote transfer"]
  node_http_probe["HTTP probe<br/>[httpx.go]"]
  node_stream["Resumable reader<br/>[reader.go]"]
end

subgraph group_extract["Extraction"]
  node_select["Engine selection<br/>[cmds.go]"]
  node_pipe_engine["Pipe extraction<br/>[pipe.go]"]
  node_rar_engine["RAR5 extraction<br/>[rar5.go]"]
  node_rar_parser["RAR5 block parser<br/>[rar5.go]"]
end

subgraph group_recovery["Safety and recovery"]
  node_safe_paths["Path protection<br/>[path.go]"]
  node_disk["Free-space checks<br/>[diskx.go]"]
  node_checkpoint[("Resume checkpoint<br/>[state.go]")]
end

node_user(("User"))
node_remote["Archive server"]
node_bsdtar["libarchive bsdtar"]
node_destination["Extracted files"]

node_user -->|"invokes"| node_cli
node_cli -->|"routes commands"| node_commands_impl
node_commands_impl -->|"probes object"| node_http_probe
node_http_probe -->|"requests metadata"| node_remote
node_commands_impl -->|"starts get or resume"| node_select
node_select -->|"opens byte stream"| node_stream
node_stream -->|"ranges and reconnects"| node_remote
node_select -->|"streams other formats"| node_pipe_engine
node_select -->|"selects RAR5 mode"| node_rar_engine
node_rar_engine -->|"parses block chain"| node_rar_parser
node_rar_engine -->|"feeds synthesized archive"| node_bsdtar
node_pipe_engine -->|"streams archive"| node_bsdtar
node_rar_engine -->|"writes extracted entries"| node_destination
node_bsdtar -->|"extracts files"| node_destination
node_rar_engine -->|"sanitizes entry paths"| node_safe_paths
node_commands_impl -->|"checks free space"| node_disk
node_rar_engine -->|"saves resume progress"| node_checkpoint
node_commands_impl -->|"loads resume state"| node_checkpoint
node_commands_impl -->|"shows checkpoint"| node_status
node_commands_impl -->|"reports transfer progress"| node_progress

click node_cli "https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/main.go"
click node_commands_impl "https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/cmds.go"
click node_http_probe "https://github.com/realkaiviljoen/streamx/blob/main/internal/httpx/httpx.go"
click node_stream "https://github.com/realkaiviljoen/streamx/blob/main/internal/httpx/reader.go"
click node_select "https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/cmds.go"
click node_pipe_engine "https://github.com/realkaiviljoen/streamx/blob/main/internal/extract/pipe.go"
click node_rar_engine "https://github.com/realkaiviljoen/streamx/blob/main/internal/extract/rar5.go"
click node_rar_parser "https://github.com/realkaiviljoen/streamx/blob/main/internal/rar5/rar5.go"
click node_safe_paths "https://github.com/realkaiviljoen/streamx/blob/main/internal/safe/path.go"
click node_disk "https://github.com/realkaiviljoen/streamx/blob/main/internal/diskx/diskx.go"
click node_checkpoint "https://github.com/realkaiviljoen/streamx/blob/main/internal/state/state.go"
click node_status "https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/cmds.go"
click node_progress "https://github.com/realkaiviljoen/streamx/blob/main/internal/ui/progress.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_commands_impl,node_status,node_progress,node_user toneBlue
class node_http_probe,node_stream toneAmber
class node_select,node_pipe_engine,node_rar_engine,node_rar_parser,node_remote toneMint
class node_safe_paths,node_disk,node_checkpoint toneRose
class node_bsdtar,node_destination toneIndigo
```

## Components

### CLI commands

- [Command router](https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/main.go): `cmd/streamx/main.go`
- [Probe and run](https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/cmds.go): `cmd/streamx/cmds.go`
- [Status inspection](https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/cmds.go): `cmd/streamx/cmds.go`
- [Progress display](https://github.com/realkaiviljoen/streamx/blob/main/internal/ui/progress.go): `internal/ui/progress.go`

### Remote transfer

- [HTTP probe](https://github.com/realkaiviljoen/streamx/blob/main/internal/httpx/httpx.go): `internal/httpx/httpx.go`
- [Resumable reader](https://github.com/realkaiviljoen/streamx/blob/main/internal/httpx/reader.go): `internal/httpx/reader.go`

### Extraction

- [Engine selection](https://github.com/realkaiviljoen/streamx/blob/main/cmd/streamx/cmds.go): `cmd/streamx/cmds.go`
- [Pipe extraction](https://github.com/realkaiviljoen/streamx/blob/main/internal/extract/pipe.go): `internal/extract/pipe.go`
- [RAR5 extraction](https://github.com/realkaiviljoen/streamx/blob/main/internal/extract/rar5.go): `internal/extract/rar5.go`
- [RAR5 block parser](https://github.com/realkaiviljoen/streamx/blob/main/internal/rar5/rar5.go): `internal/rar5/rar5.go`

### Safety and recovery

- [Path protection](https://github.com/realkaiviljoen/streamx/blob/main/internal/safe/path.go): `internal/safe/path.go`
- [Free-space checks](https://github.com/realkaiviljoen/streamx/blob/main/internal/diskx/diskx.go): `internal/diskx/diskx.go`
- [Resume checkpoint](https://github.com/realkaiviljoen/streamx/blob/main/internal/state/state.go): `internal/state/state.go`

### Other components

- **User**
- **Archive server**
- **libarchive bsdtar**
- **Extracted files**

## Connections

- User → Command router: invokes (evidence: `README.md`)
- Command router → Probe and run: routes commands (evidence: `cmd/streamx/main.go`)
- Probe and run → HTTP probe: probes object (evidence: `README.md`)
- HTTP probe → Archive server: requests metadata (evidence: `README.md`)
- Probe and run → Engine selection: starts get or resume (evidence: `cmd/streamx/cmds.go`)
- Engine selection → Resumable reader: opens byte stream (evidence: `README.md`)
- Resumable reader → Archive server: ranges and reconnects (evidence: `README.md`)
- Engine selection → Pipe extraction: streams other formats (evidence: `README.md`)
- Engine selection → RAR5 extraction: selects RAR5 mode (evidence: `README.md`)
- RAR5 extraction → RAR5 block parser: parses block chain (evidence: `internal/extract/rar5.go`)
- RAR5 extraction → libarchive bsdtar: feeds synthesized archive (evidence: `internal/extract/rar5.go`)
- Pipe extraction → libarchive bsdtar: streams archive (evidence: `README.md`)
- RAR5 extraction → Extracted files: writes extracted entries (evidence: `README.md`)
- libarchive bsdtar → Extracted files: extracts files (evidence: `README.md`)
- RAR5 extraction → Path protection: sanitizes entry paths (evidence: `internal/extract/rar5.go`)
- Probe and run → Free-space checks: checks free space (evidence: `cmd/streamx/cmds.go`)
- RAR5 extraction → Resume checkpoint: saves resume progress (evidence: `internal/extract/rar5.go`)
- Probe and run → Resume checkpoint: loads resume state (evidence: `README.md`)
- Probe and run → Status inspection: shows checkpoint (evidence: `README.md`)
- Probe and run → Progress display: reports transfer progress (evidence: `README.md`)
