How factdroid/rds works
Project type and purpose
• This is an open-source Python package and CLI tool (“syft-rds”) that layers a Remote Data Science (RDS) service on top of SyftBox.
• It provides:
• A server component (app, routers, RPC endpoints) that runs on a data-owner’s datasite
• A client library/CLI that lives in a data-scientist’s environment (local_store, rds_client)
• Notebook-based demos (quickstart, internal, API) and a BigQuery example
• Use cases: allow data scientists to execute compute on remote data (owner’s side) without exposing raw data.
File structure
• Top-level folders:
• bigquery/ → example service to publish/query Google BigQuery
• notebooks/ → Jupyter demos: quickstart, API, internal
• runtimes/ → Dockerfile and entrypoint for containerized Python runtime
• syft-rds/ → main package: CLI, client, server, orchestration, store, runtime, utils
• tests/ → unit, integration, end-to-end tests
• CI configs, justfile → build, install, run scripts
• In syft-rds/src/syft_rds:
• cli.py → entrypoint for CLI
• client/ → client APIs, connection, local_store abstractions, RPC
• server/ → FastAPI-style app, routers for job/runtime/user_code
• store/ → low-level store interface
• syft_runtime/ → code that launches the remote execution runtime
• orchestra.py → glue between client, server, store
• External dependencies: uv framework, just, uv.lock (dependency lockfile).
README for additional insights
• “full example with SyftBox” where you launch:
1) SyftBox server
2) SyftBox clients (data owner + data scientist)
3) RDS app on data-owner datasite
4) (opt.) RDS app on data-scientist datasite
5) Run notebooks
Important details
• Orchestration layer (orchestra.py) as the glue
• Test harness (tests/) as a separate lane or note to illustrate coverage
• BigQuery example as an optional external service