How ciroh-ua/ngen-datastream works
Main Components
a) Core System Components:
• Data Collection Service
• Forcing Processor
• NextGen Configuration Generator
• NextGen Execution Orchestrator (NGIAB)
• Output Handler
b) External Components:
• NextGen Water Modeling Framework (as an external dependency)
• National Water Model (data source)
• Hydrofabric Data Source
Specific Implementation Details
Create distinct sections for:
a) Input Processing:
• Hydrofabric file handling
• Forcing data collection
• Configuration generation
b) Execution Environment:
• Docker containerization (docker/ directory)
• Multi-process execution support
c) AWS Integration:
• Lambda functions (from terraform/lambda_functions/)
• Infrastructure automation
Create clear boundaries between:
• Local Execution Environment
• Cloud Infrastructure
• Data Processing Pipeline
• Model Execution Layer
Key Technical Elements to Highlight:
• BMI Configuration Generation
• Multi-model Support
• Parallel Processing
• Infrastructure as Code (Terraform)
• Container Orchestration
• Lambda Function Integration
Remember to include version information and data flow direction indicators, as this system integrates with various external components and data sources.