# fyoorer/shadowclone architecture

> Architecture diagram of the GitHub repository fyoorer/shadowclone, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fyoorer/shadowclone
- Repository: https://github.com/fyoorer/shadowclone

## Overview

ShadowClone is a CLI for splitting line-oriented input, distributing command executions across serverless functions, and collecting their output. Its documented workflow supports an additional unsplit file and cloud-provider-agnostic execution through Lithops. The excerpts show the CLI, chunk handling, object-storage operations, and worker commands in one source file; unsampled code is not expanded into guessed subsystems.

Main entry points: [`shadowclone.py`](https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py) (CLI orchestration), [`hasher.py`](https://github.com/fyoorer/shadowclone/blob/main/hasher.py) (Chunk hashing), [`config.py`](https://github.com/fyoorer/shadowclone/blob/main/config.py) (Runtime config)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli["CLI workflow"]
  node_cli["CLI orchestration<br/>[shadowclone.py]"]
  node_split["Input splitting<br/>[shadowclone.py]"]
  node_output["Result output<br/>[shadowclone.py]"]
end

subgraph group_distributed["Distributed execution"]
  node_workers["Cloud workers<br/>[shadowclone.py]"]
  node_commands["User commands"]
  node_lithops["Lithops executor"]
end

subgraph group_state["Configuration and storage"]
  node_config["Runtime config<br/>[config.py]"]
  node_storage[("Object storage")]
  node_hashing["Chunk hashing<br/>[hasher.py]"]
end

node_user(("CLI user"))

node_user -->|"runs"| node_cli
node_cli -->|"reads settings"| node_config
node_cli -->|"splits input"| node_split
node_cli -->|"uploads chunks"| node_storage
node_cli -->|"maps workers"| node_lithops
node_lithops -->|"dispatches"| node_workers
node_workers -->|"downloads input"| node_storage
node_workers -->|"runs"| node_commands
node_cli -->|"writes results"| node_output
node_cli -->|"hashes chunks"| node_hashing
node_hashing -->|"deduplicates uploads"| node_storage

click node_cli "https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py"
click node_split "https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py"
click node_output "https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py"
click node_workers "https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py"
click node_config "https://github.com/fyoorer/shadowclone/blob/main/config.py"
click node_hashing "https://github.com/fyoorer/shadowclone/blob/main/hasher.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_split,node_output,node_user toneBlue
class node_workers,node_commands,node_lithops toneAmber
class node_config,node_storage,node_hashing toneMint
```

## Components

### CLI workflow

- [CLI orchestration](https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py): `shadowclone.py`
- [Input splitting](https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py): `shadowclone.py`
- [Result output](https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py): `shadowclone.py`

### Distributed execution

- [Cloud workers](https://github.com/fyoorer/shadowclone/blob/main/shadowclone.py): `shadowclone.py`
- **User commands**
- **Lithops executor**

### Configuration and storage

- [Runtime config](https://github.com/fyoorer/shadowclone/blob/main/config.py): `config.py`
- **Object storage**
- [Chunk hashing](https://github.com/fyoorer/shadowclone/blob/main/hasher.py): `hasher.py`

### Other components

- **CLI user**

## Connections

- CLI user → CLI orchestration: runs (evidence: `README.md`)
- CLI orchestration → Runtime config: reads settings (evidence: `shadowclone.py`)
- CLI orchestration → Input splitting: splits input (evidence: `shadowclone.py`)
- CLI orchestration → Object storage: uploads chunks (evidence: `shadowclone.py`)
- CLI orchestration → Lithops executor: maps workers (evidence: `shadowclone.py`)
- Lithops executor → Cloud workers: dispatches (evidence: `shadowclone.py`)
- Cloud workers → Object storage: downloads input (evidence: `shadowclone.py`)
- Cloud workers → User commands: runs (evidence: `shadowclone.py`)
- CLI orchestration → Result output: writes results (evidence: `shadowclone.py`)
- CLI orchestration → Chunk hashing: hashes chunks (evidence: `shadowclone.py`)
- Chunk hashing → Object storage: deduplicates uploads
