How psubhajit042-tech/scengen works
ScenGen automates scenario-driven Android GUI tests: an engineer selects an app and scenario, the agent observes the screen, reasons about an action, executes it through ADB, checks the result, and records replayable output. UIED and PaddleOCR supply widgets; a local Ollama-served Qwen2.5-VL handles action decisions and visual checks. The graph represents the root testbot/ runtime; the README says scen/testbot/ is a parallel copy. The OCR lab is a separate documented experimentation lane. Some subsystem links are documented or indexed rather than shown in full excerpts.
Main entry points: core.py TestAgent FSM · memory.py Run memory · ip_region_proposal.py Component proposal · observer.py Screen observer
Components 30
Test orchestration
- TestAgent FSM
testbot/core.py - App scenarios
testbot/config.py - Screen observer
testbot/roles/observer.py - Action decider
testbot/roles/decider.py - Action executor
testbot/roles/executor.py - Step supervisor
testbot/roles/supervisor.py - Test recorder
testbot/roles/recorder.py
Screen perception
- Widget detector
testbot/uied/detect.py - Component proposal
testbot/uied/detect_compo/ip_region_proposal.py - Image detection
testbot/uied/detect_compo/lib_ip/ip_detection.py - Image preprocessing
testbot/uied/detect_compo/lib_ip/ip_preprocessing.py - Component model
testbot/uied/detect_compo/lib_ip/Component.py - Bounding boxes
testbot/uied/detect_compo/lib_ip/Bbox.py - Detection drawing
testbot/uied/detect_compo/lib_ip/ip_draw.py - Detection files
testbot/uied/detect_compo/lib_ip/file_utils.py - OCR text detection
testbot/uied/detect_text/text_detection.py - PaddleOCR adapter
testbot/uied/detect_text/ocr.py - Text elements
testbot/uied/detect_text/Text.py - Widget merge
testbot/uied/detect_merge/merge.py - Merged element model
testbot/uied/detect_merge/Element.py
VLM reasoning
- Action prompts
testbot/prompt/decider.py - Check prompts
testbot/prompt/supervisor.py - Ollama VLM client
testbot/llm.py
Device and records
- Run memory
testbot/memory.py - ADB device control
testbot/device.py - Runtime logging
testbot/logger.py
OCR experiments
- OCR benchmark and tuning
ocr_lab/scripts/run_pipeline.py
Other components
- Test operator
- Ollama Qwen VLM
- Android device
Connections 37
- Test operator → TestAgent FSM: starts test
- TestAgent FSM → App scenarios: loads scenarios
testbot/core.py - TestAgent FSM → Screen observer: coordinates
testbot/core.py - TestAgent FSM → Action decider: coordinates
testbot/core.py - TestAgent FSM → Action executor: coordinates
testbot/core.py - TestAgent FSM → Step supervisor: coordinates
testbot/core.py - TestAgent FSM → Test recorder: coordinates
testbot/core.py - TestAgent FSM → Run memory: uses run state
testbot/core.py - TestAgent FSM → ADB device control: manages device
testbot/core.py - Screen observer → ADB device control: captures screen
testbot/roles/observer.py - Screen observer → Widget detector: extracts widgets
testbot/roles/observer.py - Screen observer → Run memory: stores observations
- Widget detector → Component proposal: detects components
testbot/uied/detect.py - Widget detector → OCR text detection: detects text
testbot/uied/detect.py - Widget detector → Widget merge: merges detections
testbot/uied/detect.py - Component proposal → Image preprocessing: preprocesses image
testbot/uied/detect_compo/ip_region_proposal.py - Component proposal → Image detection: finds regions
testbot/uied/detect_compo/ip_region_proposal.py - Component proposal → Detection drawing: draws regions
testbot/uied/detect_compo/ip_region_proposal.py - Component proposal → Detection files: saves results
testbot/uied/detect_compo/ip_region_proposal.py - Image detection → Component model: builds components
testbot/uied/detect_compo/lib_ip/ip_detection.py - Component model → Bounding boxes: uses bounds
testbot/uied/detect_compo/lib_ip/Component.py - OCR text detection → PaddleOCR adapter: recognizes text
testbot/uied/detect_text/text_detection.py - OCR text detection → Text elements: creates text elements
testbot/uied/detect_text/text_detection.py - Widget merge → Merged element model: builds merged elements
testbot/uied/detect_merge/merge.py - Action decider → Action prompts: builds prompts
testbot/roles/decider.py - Action decider → Ollama VLM client: requests decisions
testbot/roles/decider.py - Action decider → Run memory: reads run context
testbot/roles/decider.py - Step supervisor → Check prompts: builds check prompts
testbot/roles/supervisor.py - Step supervisor → Ollama VLM client: checks screenshots
testbot/roles/supervisor.py - Step supervisor → Run memory: reads run context
testbot/roles/supervisor.py - Ollama VLM client → Ollama Qwen VLM: sends multimodal requests
- Action executor → ADB device control: issues actions
testbot/roles/executor.py - Action executor → Run memory: updates action history
testbot/roles/executor.py - ADB device control → Android device: controls via ADB
- Test recorder → Run memory: records history
testbot/roles/recorder.py - Test recorder → Ollama VLM client: exports chat logs
testbot/roles/recorder.py - OCR benchmark and tuning → PaddleOCR adapter: tests OCR models