How rohithcherukuri816/ai-powered-creative-intelligence-platform works
Project Type and Purpose
• This is a full-stack, monorepo web application (“AI-Powered Creative Intelligence Platform”) that lets users sketch on a canvas in the browser, send their doodles plus a text prompt to an AI‐powered backend, and receive back stylized, high-quality images generated via ControlNet + Stable Diffusion (with a PIL/Pillow fallback). There is also an alternate “colab_backend” implementation for running the same inference pipeline in Google Colab.
High-Level File Structure Analysis
• Root
– package.json, pnpm-workspace.yaml: Defines a workspace that includes “frontend” and “backend”
– scripts/: Setup helpers for installing models and env files
• frontend/
– React + Vite + Tailwind project
– src/components: DrawingCanvas, PromptInput, GeneratedImageCard, etc.
– api.js: Fetch wrapper for talking to backend
• backend/
– FastAPI application (main.py) with routes in routes/design.py
– config/: ai_config.py for model settings
– utils/image_processor.py: orchestrates preprocessing, AI call, fallback
– requirements.txt: PyTorch, FastAPI, Pillow, ControlNet/Stable Diffusion packages
• colab_backend/
– Alternate lightweight FastAPI (or Flask) app plus Jupyter notebook for Colab demos
• Dev tooling
– dev.js, setup scripts, environment examples
Key Components
1. Browser / User Interface
• React App (DrawingCanvas, PromptInput, etc.)
2. Frontend Dev Server
• Vite dev server (port 5173)
3. API Layer
• REST endpoints in FastAPI (port 8000)
4. Core Backend
• main.py → routes/design.py
• image_processor pipeline
• config (ai_config.py)
5. AI Inference
• ControlNet + Stable Diffusion (via PyTorch)
• PIL/Pillow fallback path
6. Optional Colab Microservice
• colab_backend/app.py + recognizer/processor modules
7. External Services
• Hugging Face model hub (for token-authenticated model downloads)
8. Local File Store (temporary images) or In-Memory Buffers
Relationships and Data Flow
• User draws on canvas in React → on “Generate” button, api.js posts image data + prompt to FastAPI /design route
• FastAPI route invokes image_processor:
• Applies pre-processing (skeleton extraction),
• Routes to AI inference (ControlNet + Stable Diffusion) or fallback
• Returns generated image URL/base64 back to frontend
• Frontend receives response, updates UI (GeneratedImageCard)
• Model downloads from Hugging Face occur during backend startup or scripts/setup-ai.py
Architectural Patterns & Principles
• Monorepo with clear separation: frontend vs. backend
• Pipeline pattern in backend image processing (preprocess → AI → postprocess)
• Config-driven design (ai_config.py + .env files) for environment flexibility (CPU vs GPU)
• Microservice option: colab_backend for demonstration or lightweight deployment
Putting It All Together
Tier 1 (Client): Browser/React → Tier 2 (API): FastAPI → Tier 3 (Processing): AI Engine + Fallback + External Model Hub
• Nest colab_backend as a side‐branch under Tier 3 for optional demo use
• Data flows (POST, processing pipeline, GET/response)