How artlens-turn-every-frame-into-fine-art/artlens-frontend works
1. ArtLens is a React Native mobile app built with Expo and Expo Router. The runtime is organized around file-based navigation in app/, with separate tabbed areas for camera, gallery, home, styles, and settings, plus dedicated flow screens for editing, refinement, background generation, export, and sharing.
2. The app’s main architecture is split into three layers: screen/navigation composition in app/, domain logic in core/ and features/, and shared state/UI/utilities in shared/. This separation suggests the screens stay thin while most ML, hardware, and job orchestration lives outside the route files.
3. Real-time camera processing is centered in core/camera/useFrameProcessor.ts and core/inference/InferenceEngine.ts. The camera tab likely captures frames through Vision Camera, feeds them into inference, and uses the result for live style-transfer and segmentation-driven rendering.
4. The inference subsystem is designed for performance and device constraints. core/inference/TiledInferenceRunner.ts implies large images or frames are processed in tiles to control memory use and maintain responsiveness, while core/storage/ModelManager.ts handles model availability and loading.
5. Hardware adaptation is a first-class concern. core/hardware/HardwareProfiler.ts and shared stores for hardware and battery state indicate the app adjusts its behavior based on device capability and power conditions, likely influencing frame rate, model choice, or whether certain effects are enabled.
6. Rendering is split from inference. features/canvas/SkiaRenderer.ts indicates Skia is used to compose the final visual output, while shared/ui/ImageCompareSlider.tsx supports before/after inspection in editing or refinement flows. This keeps ML output generation separate from on-device presentation.
7. User-driven style work is modeled as jobs. features/style-transfer/StyleJobService.ts and shared/stores/useStyleJobStore.ts suggest an asynchronous pipeline for launching, tracking, and persisting style-transfer tasks across screens such as edit, refine, and export.
8. Background generation is a separate capability from local style transfer. The README and app/(screens)/background-generator.tsx indicate the app calls an external generative service through services/api.ts, likely sending prompts and receiving generated backgrounds that are then combined with foreground segmentation results.
9. Image intake is unified through services/useIncomingImage.ts and features/upload/hooks/useImageSelection.ts. These modules appear to normalize images arriving from gallery picks, shares, or other entry points before handing them into the editing pipeline. The dedicated tests around useIncomingImage show this is a critical boundary with bug-prone edge cases.
10. Shared state is implemented with lightweight stores in shared/stores/, including battery, hardware profile, model, and style-job state. This suggests screen-to-screen coordination happens via global client state rather than a centralized server session, which fits an offline-capable, device-centric mobile app.
11. context/ContactContext.tsx and app/(screens)/about-contact.tsx represent a small auxiliary subsystem for user contact or support-related data, separate from the core ML workflow.
12. The repository also includes CI/CD and release automation via GitHub Actions in .github/workflows/, likely tied to EAS/Expo build and deployment for dev and prod channels. That matters architecturally because the app is shipped as a mobile build pipeline rather than as a web deployment.
13. Overall data flow: image or camera frame enters from the camera/gallery/share surfaces, is normalized by incoming-image and selection services, processed by hardware-aware inference and model management, rendered through Skia/canvas, optionally refined with background generation, and finally exported or shared from the corresponding screens.
Main entry points: (screens) Task screens · edit-canvas.tsx Edit canvas · InferenceEngine.ts Inference engine · refine.tsx Refine
Components 26
App routes
Expo Router navigation, tabs, and task screens.
- Router shell · Top-level navigation structure and route composition.
app/_layout.tsx - Tab routes · Tabbed entry points for camera, gallery, home, styles, and settings.
app/(tabs) - Task screens · Editing, refinement, background generation, export, sharing, and support flows.
app/(screens) - Camera · Live capture entry point.
app/(tabs)/camera.tsx - Gallery · Library and intake entry point.
app/(tabs)/gallery.tsx - Edit canvas · Primary editing surface for composed output.
app/(screens)/edit-canvas.tsx - Refine · Post-processing and adjustment screen.
app/(screens)/refine.tsx - Background gen · Background generation screen backed by external service.
app/(screens)/background-generator.tsx - Export · Final save and sharing handoff.
app/(screens)/export.tsx
Core engine
Camera capture, inference, hardware profiling, and model storage.
- Frame processor · Per-frame hook that prepares live camera frames for inference.
core/camera/useFrameProcessor.ts - Inference engine · Executes on-device model inference for live and edited imagery.
core/inference/InferenceEngine.ts - Tiled runner · Splits large images or frames into tiles to fit device memory limits.
core/inference/TiledInferenceRunner.ts - Model manager · Tracks model availability, loading, and readiness.
core/storage/ModelManager.ts - Hardware profile · Captures device capability data used to tune runtime behavior.
core/hardware/HardwareProfiler.ts
Feature services
Rendering, style jobs, upload, and battery gating.
- Skia renderer · Composes the final visual result from inference output and source imagery.
features/canvas/SkiaRenderer.ts - Style jobs · Coordinates style-transfer job creation, tracking, and persistence.
features/style-transfer/StyleJobService.ts - Image select · Normalizes gallery and selection inputs before editing.
features/upload/hooks/useImageSelection.ts - Battery guard · Limits or adapts expensive work based on battery state.
features/battery/useBatteryGuard.ts
Shared layer
Global stores, UI primitives, and utilities.
- Battery store · Shared battery state for coordinated power-aware behavior.
shared/stores/useBatteryStore.ts - Hardware store · Shared device capability state used across the app.
shared/stores/useHardwareProfileStore.ts - Model store · Shared model readiness and selection state.
shared/stores/useModelStore.ts - Job store · Shared style-job lifecycle state across screens.
shared/stores/useStyleJobStore.ts - Compare slider · Before/after inspection widget for edits and refinement.
shared/ui/ImageCompareSlider.tsx
External integration
API access, incoming image normalization, contact support, and build/release automation.
- Incoming image · Unifies images arriving from gallery picks, shares, and other entry points.
services/useIncomingImage.ts - API service · External background generation and other server calls.
services/api.ts - Contact context · Auxiliary support/contact data shared with the about/contact screen.
context/ContactContext.tsx
Connections 32
- Router shell → Tab routes: routes · Defines the tabbed navigation surface.
- Router shell → Task screens: routes · Mounts the dedicated workflow screens.
- Tab routes → Camera: entry · Camera tab entry point.
- Tab routes → Gallery: entry · Gallery tab entry point.
- Task screens → Edit canvas: flow · Routes into the main editing surface.
- Task screens → Refine: flow · Routes into refinement.
- Task screens → Background gen: flow · Routes into background generation.
- Task screens → Export: flow · Routes into export and sharing.
- Camera → Frame processor: frames · Live camera frames are processed through the frame hook.
- Frame processor → Inference engine: infer · Per-frame data enters the inference runtime.
- Inference engine → Tiled runner: tile · Large inputs are split for constrained-device execution.
- Inference engine → Model manager: load · Inference depends on model availability.
- Inference engine → Hardware profile: adapt · Runtime behavior adapts to device capability.
- Hardware profile → Hardware store: persist · Profile data is shared through global state.
- Battery guard → Battery store: read · Consults shared battery state before expensive work.
- Battery guard → Inference engine: throttle · Can constrain inference intensity under low power.
- Gallery → Incoming image: ingest · Gallery and share inputs are normalized on entry.
- Incoming image → Image select: select · Selection hooks prepare the normalized image for editing.
- Incoming image → Edit canvas: open · Hands off normalized images to the editing flow.
- Edit canvas → Skia renderer: render · Edit surface composes the visible output.
- Edit canvas → Compare slider: inspect · Supports before/after comparison.
- Refine → Skia renderer: render · Refinement updates the rendered result.
- Refine → Compare slider: inspect · Uses comparison UI for evaluation.
- Background gen → API service: request · Calls the external generation backend.
- Background gen → Skia renderer: compose · Generated background is combined with local foreground rendering.
- Style jobs → Job store: track · Job lifecycle is shared across screens.
- Edit canvas → Style jobs: start · Editing can launch style-transfer jobs.
- Refine → Style jobs: continue · Refinement observes and updates style jobs.
- Export → Style jobs: finish · Export finalizes completed work.
- Contact context → Task screens: support · Supports the about/contact screen within task flows.
- Battery guard → Battery store: monitor · Battery state is shared for coordinated gating.
- Model manager → Model store: share · Model readiness is exposed through global state.