# sachin210506/instant-drag architecture

> Architecture diagram of the GitHub repository sachin210506/instant-drag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sachin210506/instant-drag
- Repository: https://github.com/sachin210506/instant-drag

## Overview

#### Purpose and Runtime Topology
InstantDrag is a local interactive, drag-based image-editing system. A React/Vite client collects an uploaded image and user-drawn motion vectors; a FastAPI service runs InstantDrag inference; a separate Flask service handles authentication, registration, and text refinement. It also contains an offline video-to-training-data pipeline.

Entry points: `instadrag-frontend/src/main.jsx`, `backend_instadrag/api_server.py`, `backend_instadrag/app.py`, and the three root `run_*.bat` launchers.

#### Frontend Interaction Layer
The React application owns image upload, drag-vector drawing, login, and presentation of generated edits. The application shell and page-level routing/UI are centered in `instadrag-frontend/src/App.jsx`, `instadrag-frontend/src/Login.jsx`, and `instadrag-frontend/src/main.jsx`. It communicates with two local backend boundaries: FastAPI (normally port 8001) for image editing/saved images and Flask (port 5000) for auth/text refinement.

#### FastAPI Editing API
`backend_instadrag/api_server.py` is the serving boundary for drag-edit requests. It parses the source image and drag coordinates, converts user intent into the model’s conditioning representation, invokes ML inference, and returns the edited image to the frontend. This service should remain isolated from UI concerns and is the primary integration point for inference clients.

#### InstantDrag Inference Pipeline
`backend_instadrag/ml_pipeline.py` encapsulates FlowGen plus FlowDiffusion inference. The core flow is: sparse two-channel drag map → FlowGen dense optical flow prediction → FlowDiffusion flow-conditioned image generation. Model artifacts are external, expected under `backend_instadrag/weights/`; official InstantDrag source is resolved through `INSTANTDRAG_ROOT` or supported local clone locations. CUDA-enabled PyTorch is the intended production runtime; fallback behavior supports development without weights or GPU.

#### Auth, Refinement, and Persistence
The Flask application in `backend_instadrag/app.py` is independently deployed from the inference API and provides login/register plus text-refinement functionality. Persistence is modeled and initialized through `backend_instadrag/models.py`, `backend_instadrag/db.py`, and `backend_instadrag/init_db.py`. SQLite is the default local store, while `DATABASE_URL` enables PostgreSQL, creating a clear environment-dependent database boundary.

#### Dataset Generation Pipeline
`dataset_pipeline/` is an offline training-data producer rather than an online dependency. It derives InstantDrag-style samples from video: frame extraction, frame-pair selection, dense flow/motion-mask preparation, and sparse drag-flow creation. The principal stages are represented by `dataset_pipeline/scripts/extract_frames.py`, `dataset_pipeline/scripts/make_pairs.py`, and `dataset_pipeline/scripts/create_sparse_flow.py`; sample assembly occurs in `dataset_pipeline/scripts/assemble_sample.py`. Outputs include source/target images, masks, dense `flow.npy`, and simulated user-condition `sparse_flow.npy`.

#### Operational Dependencies
Frontend runs on Vite, typically port 5173/5174. Backend Python dependencies are scoped in `backend_instadrag/requirements.txt`; frontend dependencies in `instadrag-frontend/package.json`. External material dependencies are Hugging Face-hosted InstantDrag weights, a locally accessible official InstantDrag codebase, CUDA/PyTorch for practical inference, and optionally PostgreSQL. Local startup is coordinated by `run_flask_auth.bat`, `run_instantdrag_api.bat`, and `run_frontend.bat`.

Main entry points: [`instadrag-frontend/src/App.jsx`](https://github.com/sachin210506/instant-drag/blob/main/instadrag-frontend/src/App.jsx) (Interaction UI), [`backend_instadrag/api_server.py`](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/api_server.py) (Drag-edit API), [`run_frontend.bat`](https://github.com/sachin210506/instant-drag/blob/main/run_frontend.bat) (Local launchers), [`backend_instadrag/ml_pipeline.py`](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/ml_pipeline.py) (InstantDrag inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["React Client"]
  node_frontend_main["Vite entry<br/>React entrypoint<br/>[main.jsx]"]
  node_frontend_app["Interaction UI<br/>React application<br/>[App.jsx]"]
  node_login_ui["Login UI<br/>React page<br/>[Login.jsx]"]
end

subgraph group_editing["Editing Service"]
  node_fastapi{{"Drag-edit API<br/>FastAPI service<br/>[api_server.py]"}}
  node_conditioning["Drag conditioning<br/>sparse flow map<br/>[api_server.py]"]
  node_inference["InstantDrag inference<br/>ML pipeline<br/>[ml_pipeline.py]"]
  node_flowgen["FlowGen<br/>flow prediction<br/>[ml_pipeline.py]"]
  node_flowdiffusion["FlowDiffusion<br/>image generation<br/>[ml_pipeline.py]"]
  node_model_assets["Weights &amp; official source<br/>external ML assets"]
end

subgraph group_auth["Auth &amp; Refinement"]
  node_flask{{"Auth/refinement API<br/>Flask service<br/>[app.py]"}}
  node_data_models["Persistence models<br/>database models<br/>[models.py]"]
  node_db_setup["Database setup<br/>database initialization<br/>[db.py]"]
  node_database[("User database<br/>SQLite / PostgreSQL")]
end

subgraph group_dataset["Offline Dataset Pipeline"]
  node_extract_frames["Frame extraction<br/>dataset stage<br/>[extract_frames.py]"]
  node_make_pairs["Frame-pair selection<br/>dataset stage<br/>[make_pairs.py]"]
  node_sparse_flow["Motion and sparse flow<br/>dataset stage"]
  node_assemble_sample["Sample assembly<br/>dataset stage<br/>[assemble_sample.py]"]
end

node_launchers["Local launchers<br/>batch launchers<br/>[run_frontend.bat]"]

node_launchers -->|"starts"| node_frontend_main
node_launchers -->|"starts"| node_fastapi
node_launchers -->|"starts"| node_flask
node_frontend_main -->|"mounts"| node_frontend_app
node_frontend_app -->|"presents"| node_login_ui
node_frontend_app -->|"drag-edit requests"| node_fastapi
node_frontend_app -->|"login and refinement"| node_flask
node_fastapi -->|"builds"| node_conditioning
node_conditioning -->|"sparse flow"| node_inference
node_inference -->|"predicts via"| node_flowgen
node_flowgen -->|"dense flow"| node_flowdiffusion
node_flowdiffusion -->|"edited image"| node_fastapi
node_model_assets -.->|"models and source"| node_inference
node_flask -->|"uses"| node_data_models
node_data_models -->|"initialized through"| node_db_setup
node_db_setup -->|"connects"| node_database
node_extract_frames -->|"frames"| node_make_pairs
node_make_pairs -->|"source/target pairs"| node_sparse_flow
node_sparse_flow -->|"flow and masks"| node_assemble_sample

click node_launchers "https://github.com/sachin210506/instant-drag/blob/main/run_frontend.bat"
click node_frontend_main "https://github.com/sachin210506/instant-drag/blob/main/instadrag-frontend/src/main.jsx"
click node_frontend_app "https://github.com/sachin210506/instant-drag/blob/main/instadrag-frontend/src/App.jsx"
click node_login_ui "https://github.com/sachin210506/instant-drag/blob/main/instadrag-frontend/src/Login.jsx"
click node_fastapi "https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/api_server.py"
click node_conditioning "https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/api_server.py"
click node_inference "https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/ml_pipeline.py"
click node_flowgen "https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/ml_pipeline.py"
click node_flowdiffusion "https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/ml_pipeline.py"
click node_flask "https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/app.py"
click node_data_models "https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/models.py"
click node_db_setup "https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/db.py"
click node_extract_frames "https://github.com/sachin210506/instant-drag/blob/main/dataset_pipeline/scripts/extract_frames.py"
click node_make_pairs "https://github.com/sachin210506/instant-drag/blob/main/dataset_pipeline/scripts/make_pairs.py"
click node_sparse_flow "https://github.com/sachin210506/instant-drag/blob/main/dataset_pipeline/scripts/create_sparse_flow.py"
click node_assemble_sample "https://github.com/sachin210506/instant-drag/blob/main/dataset_pipeline/scripts/assemble_sample.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend_main,node_frontend_app,node_login_ui toneBlue
class node_fastapi,node_conditioning,node_inference,node_flowgen,node_flowdiffusion,node_model_assets toneAmber
class node_flask,node_data_models,node_db_setup,node_database toneMint
class node_extract_frames,node_make_pairs,node_sparse_flow,node_assemble_sample toneRose
class node_launchers toneNeutral
```

## Components

### React Client

- [Vite entry](https://github.com/sachin210506/instant-drag/blob/main/instadrag-frontend/src/main.jsx): React entrypoint · `instadrag-frontend/src/main.jsx`
- [Interaction UI](https://github.com/sachin210506/instant-drag/blob/main/instadrag-frontend/src/App.jsx): React application · Upload, drag vectors, login, and edited-image presentation. · `instadrag-frontend/src/App.jsx`
- [Login UI](https://github.com/sachin210506/instant-drag/blob/main/instadrag-frontend/src/Login.jsx): React page · `instadrag-frontend/src/Login.jsx`

### Editing Service

- [Drag-edit API](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/api_server.py): FastAPI service · Accepts images and drag coordinates and returns edits. · `backend_instadrag/api_server.py`
- [Drag conditioning](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/api_server.py): sparse flow map · Converts user vectors into a two-channel sparse drag map. · `backend_instadrag/api_server.py`
- [InstantDrag inference](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/ml_pipeline.py): ML pipeline · Coordinates FlowGen and FlowDiffusion inference. · `backend_instadrag/ml_pipeline.py`
- [FlowGen](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/ml_pipeline.py): flow prediction · Predicts dense optical flow from sparse drag conditioning. · `backend_instadrag/ml_pipeline.py`
- [FlowDiffusion](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/ml_pipeline.py): image generation · Generates an edited image conditioned on dense flow. · `backend_instadrag/ml_pipeline.py`
- **Weights & official source**: external ML assets · Weights and the official InstantDrag implementation resolved locally.

### Auth & Refinement

- [Auth/refinement API](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/app.py): Flask service · Provides registration, login, and text refinement. · `backend_instadrag/app.py`
- [Persistence models](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/models.py): database models · `backend_instadrag/models.py`
- [Database setup](https://github.com/sachin210506/instant-drag/blob/main/backend_instadrag/db.py): database initialization · Selects SQLite by default or PostgreSQL through DATABASE_URL. · `backend_instadrag/db.py`
- **User database**: SQLite / PostgreSQL · Environment-dependent persistence boundary.

### Offline Dataset Pipeline

Produces training samples; not an online dependency.

- [Frame extraction](https://github.com/sachin210506/instant-drag/blob/main/dataset_pipeline/scripts/extract_frames.py): dataset stage · `dataset_pipeline/scripts/extract_frames.py`
- [Frame-pair selection](https://github.com/sachin210506/instant-drag/blob/main/dataset_pipeline/scripts/make_pairs.py): dataset stage · `dataset_pipeline/scripts/make_pairs.py`
- [Motion and sparse flow](https://github.com/sachin210506/instant-drag/blob/main/dataset_pipeline/scripts/create_sparse_flow.py): dataset stage · Prepares dense flow, masks, and simulated drag flow. · `dataset_pipeline/scripts/create_sparse_flow.py`
- [Sample assembly](https://github.com/sachin210506/instant-drag/blob/main/dataset_pipeline/scripts/assemble_sample.py): dataset stage · Writes source/target images, masks, flow.npy, and sparse_flow.npy. · `dataset_pipeline/scripts/assemble_sample.py`

### Other components

- [Local launchers](https://github.com/sachin210506/instant-drag/blob/main/run_frontend.bat): batch launchers · Starts the three local runtimes. · `run_frontend.bat`

## Connections

- Local launchers → Vite entry: starts
- Local launchers → Drag-edit API: starts
- Local launchers → Auth/refinement API: starts
- Vite entry → Interaction UI: mounts
- Interaction UI → Login UI: presents
- Interaction UI → Drag-edit API: drag-edit requests
- Interaction UI → Auth/refinement API: login and refinement
- Drag-edit API → Drag conditioning: builds
- Drag conditioning → InstantDrag inference: sparse flow
- InstantDrag inference → FlowGen: predicts via
- FlowGen → FlowDiffusion: dense flow
- FlowDiffusion → Drag-edit API: edited image
- Weights & official source → InstantDrag inference: models and source
- Auth/refinement API → Persistence models: uses
- Persistence models → Database setup: initialized through
- Database setup → User database: connects
- Frame extraction → Frame-pair selection: frames
- Frame-pair selection → Motion and sparse flow: source/target pairs
- Motion and sparse flow → Sample assembly: flow and masks