# gowtham380/hydra-runner architecture

> Architecture diagram of the GitHub repository gowtham380/hydra-runner, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gowtham380/hydra-runner
- Repository: https://github.com/gowtham380/hydra-runner

## Overview

This repository is a Google Drive–to–Hugging Face movie migration worker, with optional Modal execution. Its principal workflow authenticates to Drive, scans for video files, checks destination chunks, transfers missing data, and registers movie metadata in Supabase. The two top-level scripts are parallel copies of the worker, not separate pipeline stages. The excerpts show scanning, chunk checks, and upload/registration behavior, but omit enough orchestration code that several connections remain unverified.

Main entry points: [`ingester/modal_drive_migrator.py`](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py) (Cloud worker entry), [`modal_drive_migrator.py`](https://github.com/gowtham380/hydra-runner/blob/main/modal_drive_migrator.py) (Local worker entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Worker runtime"]
  node_local_worker["Local worker entry"]
  node_cloud_worker["Cloud worker entry"]
end

subgraph group_drive["Drive access"]
  node_credentials["Service account auth"]
  node_drive_api["Google Drive"]
  node_folder_scan["Folder video scan"]
  node_range_download["Range download"]
end

subgraph group_transfer["Chunk migration"]
  node_title_parse["Title and quality mapping"]
  node_repo_router["Dataset repo mesh"]
  node_chunk_check["Existing chunk check"]
  node_chunk_upload["Chunked commit upload"]
  node_huggingface[("Hugging Face datasets")]
end

subgraph group_catalog["Catalog registration"]
  node_metadata["Movie metadata registration"]
  node_supabase[("Supabase catalog")]
end

node_local(("Local invocation"))
node_modal(("Modal runtime"))

node_local -->|"invokes"| node_local_worker
node_local_worker -->|"runs migration"| node_folder_scan
node_modal -->|"dispatches"| node_cloud_worker
node_cloud_worker -->|"runs migration"| node_folder_scan
node_folder_scan -->|"lists files"| node_drive_api
node_cloud_worker -.->|"acquires token"| node_credentials
node_credentials -.->|"authorizes access"| node_drive_api
node_cloud_worker -->|"parses names"| node_title_parse
node_cloud_worker -->|"selects destination"| node_repo_router
node_cloud_worker -->|"checks parts"| node_chunk_check
node_chunk_check -->|"reads chunk listing"| node_huggingface
node_cloud_worker -.->|"downloads missing data"| node_range_download
node_cloud_worker -.->|"uploads missing parts"| node_chunk_upload
node_chunk_upload -->|"commits chunks"| node_huggingface
node_cloud_worker -.->|"registers movie"| node_metadata
node_metadata -->|"writes record"| node_supabase

click node_local_worker "https://github.com/gowtham380/hydra-runner/blob/main/modal_drive_migrator.py"
click node_cloud_worker "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"
click node_credentials "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"
click node_folder_scan "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"
click node_range_download "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"
click node_title_parse "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"
click node_repo_router "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"
click node_chunk_check "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"
click node_chunk_upload "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"
click node_metadata "https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_local_worker,node_cloud_worker toneBlue
class node_credentials,node_drive_api,node_folder_scan,node_range_download toneAmber
class node_title_parse,node_repo_router,node_chunk_check,node_chunk_upload,node_huggingface toneMint
class node_metadata,node_supabase toneRose
class node_local,node_modal toneIndigo
```

## Components

### Worker runtime

- [Local worker entry](https://github.com/gowtham380/hydra-runner/blob/main/modal_drive_migrator.py): `modal_drive_migrator.py`
- [Cloud worker entry](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`

### Drive access

- [Service account auth](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`
- **Google Drive**
- [Folder video scan](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`
- [Range download](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`

### Chunk migration

- [Title and quality mapping](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`
- [Dataset repo mesh](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`
- [Existing chunk check](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`
- [Chunked commit upload](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`
- **Hugging Face datasets**

### Catalog registration

- [Movie metadata registration](https://github.com/gowtham380/hydra-runner/blob/main/ingester/modal_drive_migrator.py): `ingester/modal_drive_migrator.py`
- **Supabase catalog**

### Other components

- **Local invocation**
- **Modal runtime**

## Connections

- Local invocation → Local worker entry: invokes (evidence: `modal_drive_migrator.py`)
- Local worker entry → Folder video scan: runs migration (evidence: `modal_drive_migrator.py`)
- Modal runtime → Cloud worker entry: dispatches (evidence: `ingester/modal_drive_migrator.py`)
- Cloud worker entry → Folder video scan: runs migration (evidence: `ingester/modal_drive_migrator.py`)
- Folder video scan → Google Drive: lists files (evidence: `ingester/modal_drive_migrator.py`)
- Cloud worker entry → Service account auth: acquires token (evidence: `ingester/modal_drive_migrator.py`)
- Service account auth → Google Drive: authorizes access
- Cloud worker entry → Title and quality mapping: parses names (evidence: `ingester/modal_drive_migrator.py`)
- Cloud worker entry → Dataset repo mesh: selects destination (evidence: `ingester/modal_drive_migrator.py`)
- Cloud worker entry → Existing chunk check: checks parts (evidence: `ingester/modal_drive_migrator.py`)
- Existing chunk check → Hugging Face datasets: reads chunk listing (evidence: `ingester/modal_drive_migrator.py`)
- Cloud worker entry → Range download: downloads missing data (evidence: `ingester/modal_drive_migrator.py`)
- Cloud worker entry → Chunked commit upload: uploads missing parts (evidence: `ingester/modal_drive_migrator.py`)
- Chunked commit upload → Hugging Face datasets: commits chunks (evidence: `ingester/modal_drive_migrator.py`)
- Cloud worker entry → Movie metadata registration: registers movie (evidence: `ingester/modal_drive_migrator.py`)
- Movie metadata registration → Supabase catalog: writes record (evidence: `ingester/modal_drive_migrator.py`)
