mywai-ai/i-genius

1. Overall shape

- This is a local-first Streamlit application for robot demonstration processing.

- The repository is split into a presentation layer (new_ui pages/components), a service layer that handles uploads and orchestration, and a processing layer under core/ that performs the actual CV/trajectory/DMP work.

2. Main user entrypoints

- The primary app entry is the Streamlit landing page in src/streamlit_template/new_ui/pages/Common/landing_page.py.

- The repo also keeps older/alternate UI and pipeline code under src/streamlit_template/ui and src/streamlit_template/core/*, which suggests the architecture has evolved from script-like pipelines into a more structured UI/service split.

3. UI architecture

- new_ui/pages/ contains the route-level screens for the three major workflows: BAG, SVO, and Generic.

- new_ui/components/ holds reusable UI widgets such as frame viewers, video players, and stepper bars.

- new_ui/Common/ provides shared app state, styling, config, and source definitions.

4. Service layer boundary

- new_ui/services/ is the orchestration boundary between UI and processing.

- These services manage uploads, validation, file handling, and pipeline triggers:

- zip_upload_handler.py

- frame_service.py

- video_extraction_service.py

- bag_helpers.py

- svo_helpers.py

- robot-related services for playback/action

- This layer is where incoming user data becomes structured runtime artifacts on disk.

5. Core processing pipelines

- The heavy computation lives in src/streamlit_template/core/.

- There are domain-specific workflows:

- core/SVO/ for ZED SVO/SVO2-based RGB-D processing

- core/BAG/ for ROS bag-style workflows

- core/Generic/ for broader inputs such as uploaded video

- These folders contain ordered step scripts, indicating a staged pipeline architecture rather than a single monolithic job.

6. SVO workflow: the main reusable path

- The README makes clear the most mature workflow is ZED .svo / .svo2 exported into an RGB-D ZIP.

- Expected inputs include camera intrinsics, RGB frames, metric depth maps, depth-color previews, and a video preview.

- The SVO pipeline then runs sequential steps for hand trajectory extraction, smoothing, object trajectory extraction, grasp estimation, trajectory reconstruction, segmentation, skill extraction, DMP learning/adaptation, and robot playback evaluation.

7. BAG workflow

- core/BAG/ mirrors the same high-level progression, but for bag-based demonstration data.

- It includes explicit numbered stages from frame extraction through DMP learning and skill reuse tests.

- This suggests BAG is another ingestion path into the same downstream learning and playback concepts, not a separate product line.

8. Generic / video-oriented workflow

- core/Generic/ provides a less specialized path that supports frame extraction, hand/object detection, trajectory extraction, and DMP generation.

- The README notes MP4 input is supported in the Skill-Reuse branch, but without metric depth it is less suitable for precise robot-space pick-and-place.

- Architecturally, this is a lower-fidelity input path that converges on the same trajectory/skill outputs.

9. Shared model and robot asset boundary

- data/Common/ is a shared asset layer rather than runtime output.

- It contains:

- AI models for hand and object detection

- robot descriptions, URDFs, meshes, and config files

- This boundary is important: processing code depends on these assets, but they are not generated by the app.

10. Runtime data model

- data/SVO/, data/BAG/, and data/Generic/ are the local runtime workspaces.

- Each has subfolders for uploads, downloads, frames, depth, hands, objects, segmentation, plots, videos, and debug artifacts.

- The architecture is intentionally filesystem-centric: uploads enter the workspace, pipeline steps write intermediate outputs there, and the UI reads them back for visualization.

11. External integration points

- The repo includes scripts/vulcanexus/, which is a separate integration surface for DDS / ROS 2 / Fast DDS style robot communication.

- Scripts indicate trajectory publishing/subscribing, edge executors, and WAN transport setup.

- This is the bridge from local inference/trajectory generation to robot-side execution.

12. Deployment and runtime

- The app runs locally via Streamlit, and there is a Docker path via docker-compose.yml and Dockerfile.

- supervisord.conf suggests containerized multi-process runtime management.

- The default local port in the README is 8504; Docker exposes the app on 9002.

13. Documentation as part of the system

- tutorials/ is not just reference material; it defines the expected pre-processing contract for SVO ZIP creation and custom robot packaging.

- The tutorials are effectively part of the architecture because they define the data shape that the app’s validators and pipelines expect.

14. Key architectural takeaway

- The repo is a local, file-backed, stepwise demonstration-processing system.

- Data flows from user uploads to structured workspace folders, through domain-specific pipelines, and out to learned trajectories, DMP artifacts, plots, and optional robot execution interfaces.