sachin210506/instant-drag

Purpose and Runtime Topology

InstantDrag is a local interactive, drag-based image-editing system. A React/Vite client collects an uploaded image and user-drawn motion vectors; a FastAPI service runs InstantDrag inference; a separate Flask service handles authentication, registration, and text refinement. It also contains an offline video-to-training-data pipeline.

Entry points: instadrag-frontend/src/main.jsx, backend_instadrag/api_server.py, backend_instadrag/app.py, and the three root run_*.bat launchers.

Frontend Interaction Layer

The React application owns image upload, drag-vector drawing, login, and presentation of generated edits. The application shell and page-level routing/UI are centered in instadrag-frontend/src/App.jsx, instadrag-frontend/src/Login.jsx, and instadrag-frontend/src/main.jsx. It communicates with two local backend boundaries: FastAPI (normally port 8001) for image editing/saved images and Flask (port 5000) for auth/text refinement.

FastAPI Editing API

backend_instadrag/api_server.py is the serving boundary for drag-edit requests. It parses the source image and drag coordinates, converts user intent into the model’s conditioning representation, invokes ML inference, and returns the edited image to the frontend. This service should remain isolated from UI concerns and is the primary integration point for inference clients.

InstantDrag Inference Pipeline

backend_instadrag/ml_pipeline.py encapsulates FlowGen plus FlowDiffusion inference. The core flow is: sparse two-channel drag map → FlowGen dense optical flow prediction → FlowDiffusion flow-conditioned image generation. Model artifacts are external, expected under backend_instadrag/weights/; official InstantDrag source is resolved through INSTANTDRAG_ROOT or supported local clone locations. CUDA-enabled PyTorch is the intended production runtime; fallback behavior supports development without weights or GPU.

Auth, Refinement, and Persistence

The Flask application in backend_instadrag/app.py is independently deployed from the inference API and provides login/register plus text-refinement functionality. Persistence is modeled and initialized through backend_instadrag/models.py, backend_instadrag/db.py, and backend_instadrag/init_db.py. SQLite is the default local store, while DATABASE_URL enables PostgreSQL, creating a clear environment-dependent database boundary.

Dataset Generation Pipeline

dataset_pipeline/ is an offline training-data producer rather than an online dependency. It derives InstantDrag-style samples from video: frame extraction, frame-pair selection, dense flow/motion-mask preparation, and sparse drag-flow creation. The principal stages are represented by dataset_pipeline/scripts/extract_frames.py, dataset_pipeline/scripts/make_pairs.py, and dataset_pipeline/scripts/create_sparse_flow.py; sample assembly occurs in dataset_pipeline/scripts/assemble_sample.py. Outputs include source/target images, masks, dense flow.npy, and simulated user-condition sparse_flow.npy.

Operational Dependencies

Frontend runs on Vite, typically port 5173/5174. Backend Python dependencies are scoped in backend_instadrag/requirements.txt; frontend dependencies in instadrag-frontend/package.json. External material dependencies are Hugging Face-hosted InstantDrag weights, a locally accessible official InstantDrag codebase, CUDA/PyTorch for practical inference, and optionally PostgreSQL. Local startup is coordinated by run_flask_auth.bat, run_instantdrag_api.bat, and run_frontend.bat.