How terrafloww/rasteret works
Main Components
a) Core System Components:
• STAC Interface Layer
• COG Processing Engine
• Cloud Provider Interface
• Local Cache System
• Data Output Handlers
b) External Systems:
• Cloud Storage (AWS S3)
• STAC Endpoints (Earth-search, Landsatlook)
• Local File System
Client Layer
API Layer
Processing Layer
Storage Layer
External Services Layer
Specific Component Details
a) Client Layer:
• Entry points through the main Rasteret class
• Both Collection Creation and Data Retrieval flows
b) API Layer (src/rasteret/):
• Core API components
• Type definitions
• Logging system
c) Processing Layer (src/rasteret/core/):
• Collection processor
• Scene processor
• Utility functions
d) Storage Layer:
• Local cache system
• GeoParquet storage
• COG header cache
e) External Services Layer:
• STAC endpoints
• S3 storage
• Authentication systems
Data Flow Representations
• Initial STAC metadata retrieval flow
• COG header caching flow
• Data retrieval and processing flow
• Output generation flow (Xarray/GeoDataFrame)
Key Technical Elements to Highlight
• Parallel processing capabilities
• Caching mechanism
• STAC metadata indexing
• COG tile retrieval optimization
• Output format conversions
Implementation Details
• How src/rasteret/stac/ handles STAC interactions
• How src/rasteret/cloud.py interfaces with cloud providers
• Demonstrate how src/rasteret/core/ manages processing
• Fetch module's role in data retrieval
Performance Optimization Points
Areas where the system implements its key optimizations:
• Single HTTP request per image tile
• Local header caching
• Parallel processing implementation
• STAC metadata indexing
Remember to include version information and any relevant notes about the work-in-progress nature of the library, as mentioned in the README warning.