How ciroh-ua/ngiab-cloudinfra works
Main Components
• Docker Container Environment: The “docker” directory holds the Dockerfile and startup scripts that bundle the model’s runtime environment.
• Configuration Files & Data: The “config” directory and the “docs” provide the necessary model configuration and documentation. Additionally, the “ngen-run” directory structure (as explained in the README) is central to the modeling run – containing subfolders like config, forcings, outputs, lakeout, metadata, and restart.
• Infrastructure Management: The “terraform” directory with subdirectories for “network” and “user” implies that certain network and user-related infrastructures are managed via Terraform.
• GitHub CI/CD and Automation: The “.github” directory (workflows, templates, dependabot configuration) shows use of GitHub Actions for building, testing, and actions such as pushing Docker images and sending notifications (e.g., to Slack).
• Auxiliary Scripts and Tools: Scripts like guide.sh, viewOnTethys.sh, and runTeehr.sh provide automation for starting up the process, running visualizations, and possibly integrating evaluation tools (TEEHR).
Relationships and Interactions
• User Interaction: The user downloads or clones the repository, sets up the NextGen run directory structure, and then triggers the guide script.
• Docker/Containerization: The Dockerfile builds the container image that encapsulates the NextGen runtime. The container then interacts with the run directory (ngen-run) – reading configuration files, processing input data, and managing outputs.
• Data Flow:
• Input data is downloaded (from external S3 sources as defined in the Quick Start Guide) and placed into the run directory.
• Within the container, configuration, forcing, and hydrofabric files (e.g., geopackage used for the NextGen run) are used to execute the model.
• Outputs are generated, which are later visualized either through the built-in visualization tools or via additional scripts.
• Infrastructure Automation: Terraform scripts configure networking and user settings needed for cloud deployments or for local simulation of network conditions.
• Continuous Integration/Deployment: GitHub workflows automate building and testing the Docker environment, keeping the container images updated.
Architectural Patterns and Principles
• Containerization: Encapsulation of the model runtime environment inside Docker ensures reproducibility and consistency.
• Infrastructure as Code: Use of Terraform streamlines the creation and management of the required infrastructure.
• Modular Run Directory: The separation of configuration, inputs (forcings), and outputs (with optional metadata) enforces the separation of concerns and aids in debugging and future scalability.
• Automation Pipelines: CI/CD via GitHub Actions automates testing and deployment, which is crucial for maintaining an open-source framework.
Relevant Technologies, Frameworks, and Libraries
• Docker (container image creation and runtime)
• Terraform (infrastructure provisioning)
• GitHub Actions (CI/CD pipelines for build & test, Docker builds, etc.)
• Bash/Shell Scripting (guide.sh, viewOnTethys.sh, etc.)
• Visualization tools integrated within the containerized environment
• JSON, YAML, and possibly other configuration formats for model configurations
Elements
• Clear labels for each component:
• “User/Local Machine” initiating the process.
• “Docker Engine/Container” representing the runtime environment built from docker/Dockerfile.
• “NGI Run Directory” with subfolders (config, forcings, outputs, etc.).
• “Terraform Modules” for network and user configuration.
• “Continuous Integration” elements (GitHub Actions pipelines).
• Data flow from user actions (e.g., downloading sample data) to the container’s execution process.
• Interaction between GitHub workflows and Docker builds.
• Dependencies from the NextGen configuration to the modeling outputs.
• Rectangles for components (e.g., Docker container, Terraform modules).
• Off to the side, illustrate Terraform-managed resources to show how network/user infra is set up.
• Optionally, add the visualization tool invocation (viewOnTethys.sh) as a separate process that fetches output data and displays results.