How d3b-center/flywheel-soft-data-copy works
This repository is a collection of Flywheel data-management scripts. Its principal workflow loads CBTN subject IDs, discovers matching Flywheel project/subject/session records from either the D3b warehouse or Flywheel, writes a catalog CSV, then uses that catalog to soft-copy subjects or sessions into a destination project. Separate scripts queue Flywheel’s soft-copy gear and export a project file manifest. Source excerpts show the direct calls and data handling; external service behavior is represented only at the integration boundary.
Main entry points: find_fw_data.py Discovery CLI · fw_soft_copy_gear.py Gear runner · fw_soft_copy_sdk.py SDK copy CLI · utils.py Match finder
Components 16
Data discovery
- Subject inputs
- Discovery CLI
find_fw_data.py - Subject loader
find_fw_data.py - Match finder
utils.py - D3b warehouse
- Match catalog CSV
Soft copy
- SDK copy CLI
fw_soft_copy_sdk.py - Copy planner
fw_soft_copy_sdk.py - Soft-copy operations
fw_soft_copy_sdk.py - Gear runner
fw_soft_copy_gear.py
Manifest export
- Manifest exporter
pull_az_file_manifest.py - Project DataView
- Manifest CSV
Other components
- Operator
- Flywheel service
- Soft-copy gear
Connections 23
- Operator → Discovery CLI: runs discovery
README.md - Subject inputs → Subject loader: provides IDs
find_fw_data.py - Discovery CLI → Subject loader: loads subjects
find_fw_data.py - Discovery CLI → Match finder: calls finder
find_fw_data.py - Match finder → D3b warehouse: queries records
utils.py - Match finder → Flywheel service: searches subjects
utils.py - Discovery CLI → Flywheel service: authenticates
find_fw_data.py - Discovery CLI → Match catalog CSV: writes matches
find_fw_data.py - Match catalog CSV → SDK copy CLI: supplies rows
README.md - Operator → SDK copy CLI: runs copy
README.md - SDK copy CLI → Flywheel service: reads destination
fw_soft_copy_sdk.py - SDK copy CLI → Copy planner: checks and plans
fw_soft_copy_sdk.py - Copy planner → Flywheel service: looks up source
fw_soft_copy_sdk.py - Copy planner → Soft-copy operations: dispatches copies
fw_soft_copy_sdk.py - Soft-copy operations → Flywheel service: creates soft copies
fw_soft_copy_sdk.py - Operator → Gear runner: runs gear script
fw_soft_copy_gear.py - Gear runner → Flywheel service: looks up projects
fw_soft_copy_gear.py - Gear runner → Soft-copy gear: queues jobs
fw_soft_copy_gear.py - Gear runner → Flywheel service: creates destination
fw_soft_copy_gear.py - Operator → Manifest exporter: runs export
README.md - Manifest exporter → Flywheel service: reads project view
pull_az_file_manifest.py - Flywheel service → Project DataView: provides rows
- Manifest exporter → Manifest CSV: writes manifest
pull_az_file_manifest.py