How must-canary/ai-visual-mobile-app-testing works
Visual Mobile Automation Testing is a browser playground for uploading Android APKs, managing emulator sessions, authoring plain-English steps, and observing automation runs. Its main workflow parses a script, executes each step against an Appium session, resolves described screen elements through local history or vision, and streams run events back to the UI. The graph also preserves APK/app management, saved tests, screenshots, and cache controls. Some subsystem links are documented in the README or the SOURCE INDEX; unsampled internals are not expanded beyond those supported relationships.
Main entry points: context.tsx Playground · step-executor.ts Step executor · session-manager.ts Appium sessions · route.ts APK app API
Components 25
Playground UI
- Playground
components/playground/context.tsx - Script editor
components/playground/ScriptEditor.tsx - Session controls
components/playground/SessionBar.tsx - Local-first control
components/playground/LocalFirstControl.tsx - Run event hook
hooks/useRunEvents.ts
HTTP API
- APK app API
app/api/apps/route.ts - Session API
app/api/sessions/route.ts - Run API
app/api/runs/route.ts - Run event API
app/api/runs/[id]/events/route.ts - Cache API
app/api/cache/route.ts
Automation runtime
- APK upload
lib/apk/upload.ts - APK metadata
lib/apk/badging.ts - Script parser
lib/dsl/parser.ts - Run controller
lib/engine/run-controller.ts - Step executor
lib/engine/step-executor.ts - Vision locate
lib/vision/locate.ts - Locate history
lib/engine/locate-cache.ts - Run event bus
lib/engine/events.ts - Screenshot store
lib/engine/screenshot-store.ts
Android device
- Appium sessions
lib/appium/session-manager.ts - Android SDK
lib/android/sdk.ts
Persistence
- MongoDB collections
lib/mongodb.ts
Other components
- Test author
- Vision provider
- Appium emulator
Connections 32
- Test author → Playground: authors tests
README.md - Playground → APK app API: manages APKs
components/playground/context.tsx - APK app API → APK upload: streams upload
app/api/apps/route.ts - APK app API → APK metadata: reads metadata
app/api/apps/route.ts - APK app API → MongoDB collections: stores app
app/api/apps/route.ts - Playground → Session API: manages sessions
components/playground/context.tsx - Session API → Appium sessions: starts session
app/api/sessions/route.ts - Appium sessions → Android SDK: lists devices
lib/appium/session-manager.ts - Appium sessions → Appium emulator: creates driver
lib/appium/session-manager.ts - Playground → Script editor: edits steps
README.md - Playground → Run API: submits run
README.md - Run API → Script parser: parses script
app/api/runs/route.ts - Run API → Appium sessions: gets session
app/api/runs/route.ts - Run API → Run controller: starts run
app/api/runs/route.ts - Run controller → MongoDB collections: persists run
lib/engine/run-controller.ts - Run controller → Step executor: executes steps
lib/engine/run-controller.ts - Run controller → Run event bus: publishes events
lib/engine/run-controller.ts - Step executor → Vision locate: locates targets
lib/engine/step-executor.ts - Vision locate → Vision provider: requests vision
README.md - Step executor → Locate history: reuses history
lib/engine/step-executor.ts - Step executor → Appium sessions: drives session
lib/engine/step-executor.ts - Step executor → Screenshot store: stores captures
lib/engine/step-executor.ts - Step executor → Run event bus: reports stages
lib/engine/step-executor.ts - Playground → Local-first control: controls reuse
components/playground/context.tsx - Local-first control → Cache API: reads cache stats
- Cache API → Locate history: manages cache
app/api/cache/route.ts - Playground → Run event hook: subscribes events
components/playground/context.tsx - Run event hook → Run event API: opens event stream
hooks/useRunEvents.ts - Run event API → Run event bus: streams events
app/api/runs/[id]/events/route.ts - Run event API → MongoDB collections: loads run events
app/api/runs/[id]/events/route.ts - Appium sessions → Appium emulator: closes session
- MongoDB collections → Locate history: backs cache records