How g-nitin/artedge works
Project Type and Purpose
• Type: A mobile-first iOS application (Swift/SwiftUI) for real-time Neural Style Transfer (NST) on device, accompanied by an offline Python-based model development and conversion pipeline.
• Purpose: Demonstrate edge-computing NST: train/adapt models in PyTorch, convert to Core ML, and run in an iOS app with low latency & privacy.
File Structure
• Top-Level Directories
– ArtEdge/: iOS app source – UI (ContentView, CameraPreviewView, ImagePicker), service (CameraService, StyleTransferService), assets, and Core ML models (.mlpackage).
– models/: Python code – training/inference for four NST architectures (AdaIN, FST, StyTR-2, AesFA), plus conversion scripts.
• Key Config/Project Files:
– ArtEdge.xcodeproj: Xcode project definition.
– pyproject.toml / uv.lock: Python dependency management.
– convert.py scripts – transform PyTorch models → Core ML packages.
Main Components
A. Model Development Pipeline (Offline)
1. PyTorch Training Modules (models/AdaIN, AesFA, FST, StyTR-2)
2. convert.py & Core ML Tools (TorchScript → .mlpackage)
3. Generated Core ML Models (.mlpackage files in ArtEdge/models/)
B. iOS Application (Runtime)
1. UI Layer (SwiftUI Views: ContentView, CameraPreviewView, ImagePicker)
2. Service Layer (CameraService captures input, StyleTransferService invokes Core ML)
3. Core ML Inference Engine (Apple Core ML framework + Neural Engine hardware)
5. Hardware/OS (iOS device, Neural Engine, GPU/CPU)
C. External Frameworks & Tools
• PyTorch (model development)
• Core ML Tools (model conversion)
• SwiftUI, AVFoundation (camera), Core ML (inference)
Relationships & Data Flows
• Offline Flow:
PyTorch Training → TorchScript → Core ML Tools → .mlpackage → Xcode Project
• Runtime Flow:
User taps “Select Style” → UI (ContentView) passes image to StyleTransferService → Core ML model inference → Stylized image → UI displays result
• Hardware Acceleration: StyleTransferService → Core ML framework → Neural Engine/GPU → returns output buffer
Architectural Patterns & Design Principles
• MVVM-ish / Service Pattern in SwiftUI:
– Views bound to models (StyleInfo) and call services for business logic.
• Separation of Concerns:
– Offline model development vs. on-device inference.
– UI, services, ML inference all modular.
• Edge-Computing Principle:
– All image data and inference occur locally, no network.