How opengraphlabs/tensorflowsui works
Based on our analysis of the file tree and README, this project is a full‐stack system that bridges traditional Web2 deep learning (DL) and machine learning (ML) workflows with a fully decentralized, on-chain execution environment provided via the Sui blockchain. The project’s purpose is to “democratize” AI by ensuring model transparency, reproducibility, and verifiability in a blockchain context. It does this by converting, deploying, and executing AI models (originally trained and stored in Web2 formats) as Move smart contracts while integrating off-chain data processing and blockchain provenance tracking.
• Web3.0 & Blockchain Layer:
– Components: “walrus” (Go server for blockchain interactions, digital provenance, and transaction trajectory), and parts of “tensorflowSuiLib” (which contains the Move-based core libraries for tensor and graph operations).
– Role: Handles smart contract execution, data provenance, verifiable on-chain inference, and blockchain-based state management.
• Model Publisher:
– Directory: “modelPublisher”
– Role: Processes converted models (e.g., TensorFlow.js format), transforms weights into a fixed-point representation, auto-generates Move smart contracts, deploys them on Sui, and manages model versioning.
• Model User:
– Directory: “modelUser”
– Role: Provides the CLI to perform model inference by retrieving input data (e.g., from Walrus), executing inference (in split or PTB modes), and verifying blockchain results.
• Atoma Framework:
– Directory: “atoma”
– Role: Responsible for generating and converting input data using Atoma’s LLM capabilities into a blockchain-compatible format.
• Supporting SDK and Dev Tools:
– Directories: “dev_sdk” (includes code for handling .h5 models, reading model files, and possibly test utilities) and portions of “tensorflowSuiLib” that deal with the low-level tensor/move code.
• External Components:
– Integrations like external libraries (e.g., Move Standard Library dependencies, third-party Node modules in package.json files, and Go modules in “walrus”) should be noted to illustrate dependency on the Sui blockchain network.
Relationships and Data Flow
• Data Generation and Preprocessing:
• Model Conversion & Publishing:
• On-chain Inference and User Interaction:
• Provenance and Verification:
Relevant Architectural Patterns and Design Principles
• Separation of Concerns: Each major component (data generation, model publishing, on-chain execution, inference, blockchain verification) is standalone but communicates through clearly defined interfaces.
• Modularization: Modules written in different languages (JavaScript/Node for modelPublisher and modelUser, Go for walrus, Python for dev_sdk utilities, and Move for on-chain libraries) are integrated cohesively.
• Hybrid Inference: The design shows both on-chain and off-chain processes, where inference can be performed fully on-chain for small models or via hybrid approaches (using split transactions and PTB) to optimize efficiency.
• Data & Transaction Provenance: The use of Walrus for tracking digital provenance is critical. Make sure to include that the system creates immutable, verifiable records of computations.
Details of Technologies, Frameworks, and Libraries
• Node.js and npm (for modelPublisher and modelUser scripts, package.json dependencies)
• Go (for the “walrus” server interacting with Sui)
• Move language (used for the on-chain smart contracts and core tensor operations in tensorflowSuiLib)
• TensorFlow.js and conversion tools (for converting .h5 models to a web-compatible format)
• External blockchain explorers like SuiScan (reference links in README)