How podaac/data-subscriber works
This package provides two command-line workflows for PO.DAAC data: an on-demand bulk downloader and a recurring subscriber that tracks newly ingested granules. Both search CMR and download matching data; the subscriber also persists an update timestamp and can route requests through Harmony subsetting. The sampled entry points show these orchestration paths and their calls into shared access utilities. Some transfer and state-management implementation details are in omitted excerpts, so the graph shows only supported high-level relationships.
Main entry points: podaac_access.py PO.DAAC access · podaac_data_subscriber.py Data subscriber · podaac_data_downloader.py Bulk downloader · subsetting.py Subset routing
Components 13
Download workflows
- Bulk downloader
subscriber/podaac_data_downloader.py - Data subscriber
subscriber/podaac_data_subscriber.py
Search and access
- PO.DAAC access
subscriber/podaac_access.py - Earthdata auth
subscriber/podaac_access.py - Token redaction
subscriber/token_formatter.py
Transfer and state
- Local output
Harmony subsetting
- Subset routing
subscriber/subsetting.py
Other components
- CLI user
- CMR search
- Earthdata Login
- Data archive
- Harmony jobs
- Release API
Connections 27
- CLI user → Bulk downloader: runs
- CLI user → Data subscriber: runs
- Bulk downloader → Earthdata auth: gets token
- Data subscriber → Earthdata auth: gets token
- Earthdata auth → Earthdata Login: requests tokens
- Bulk downloader → PO.DAAC access: searches CMR
- Data subscriber → PO.DAAC access: searches CMR
- PO.DAAC access → CMR search: queries collections
- Bulk downloader → PO.DAAC access: retrieves granules
- Data subscriber → PO.DAAC access: retrieves granules
- PO.DAAC access → Data archive: downloads data
- Bulk downloader → Local output: writes files
- Data subscriber → Local output: writes files
- Data subscriber → Local output: reads update state
- Data subscriber → Local output: writes update state
- Bulk downloader → Subset routing: routes subset requests
- Data subscriber → Subset routing: routes subset requests
- Subset routing → PO.DAAC access: checks support
- Subset routing → PO.DAAC access: finds or submits job
- PO.DAAC access → Harmony jobs: submits and monitors
- Subset routing → PO.DAAC access: downloads results
- PO.DAAC access → Local output: saves job state
- Bulk downloader → Token redaction: formats logs
- Data subscriber → Token redaction: formats logs
- Bulk downloader → PO.DAAC access: checks releases
- Data subscriber → PO.DAAC access: checks releases
- PO.DAAC access → Release API: checks latest release