How d3b-center/image-deid-etl works
This is a command-line ETL for discovering Orthanc studies, mapping their DICOM metadata to program-specific subject/session labels, converting and organizing anonymized NIfTI outputs, and delivering results to Flywheel or S3. The main workflow is driven by the CLI; processed-study UUIDs are tracked in a database, while validation and image-review outputs support quality control. The graph reflects the README’s command capabilities and sampled runtime code; some command wiring and unsampled implementation details are therefore left at the documented level.
Main entry points: __main__.py ETL commands · main_pipeline.py De-ID workflow · external_data_handling.py Subject mapping · custom_etl.py ETL transformations
Components 15
CLI orchestration
- ETL commands
src/image_deid_etl/image_deid_etl/__main__.py
Study processing
- DICOM inspection
src/image_deid_etl/image_deid_etl/custom_dicom.py - De-ID workflow
src/image_deid_etl/image_deid_etl/main_pipeline.py - Subject mapping
src/image_deid_etl/image_deid_etl/external_data_handling.py - ETL transformations
src/image_deid_etl/image_deid_etl/custom_etl.py
Delivery and review
- NIfTI outputs
- Review images
src/image_deid_etl/image_deid_etl/images_no_save.py - Flywheel operations
src/image_deid_etl/image_deid_etl/custom_flywheel.py
State and sources
- Orthanc client
src/image_deid_etl/image_deid_etl/orthanc.py - Processed UUIDs
src/image_deid_etl/image_deid_etl/database.py - Mapping data
Other components
- CLI user
- Orthanc
- Flywheel
- S3 backup
Connections 15
- CLI user → ETL commands: runs commands
- ETL commands → Orthanc client: checks studies
- Orthanc client → Orthanc: queries and downloads
- ETL commands → Processed UUIDs: reads or records UUIDs
- ETL commands → De-ID workflow: runs de-identification
- De-ID workflow → Subject mapping: resolves subject labels
- Subject mapping → Mapping data: reads mappings
- Subject mapping → ETL transformations: uses DICOM metadata helpers
- De-ID workflow → ETL transformations: converts and organizes
- ETL transformations → NIfTI outputs: writes NIfTI files
- De-ID workflow → Review images: generates review images
- Review images → NIfTI outputs: reads NIfTI files
- ETL commands → Flywheel operations: uploads and adds metadata
- Flywheel operations → Flywheel: uploads files and metadata
- ETL commands → S3 backup: backs up NIfTIs