How john-amal/ifs_ic_gen works
Project type & purpose (what you’re diagramming)
This repo is not an application/service architecture; it’s an HPC batch-style data processing utility: a single Bash pipeline that generates IFS atmosphere initial condition (IC) files for branching an IFS–FESOM coupled model run.
Key points from README + tree:
• Core artifact: generate_initial_conditions_parallel_only_IFS.sh (the “system”).
• Inputs: IFS restart GRIB files produced by a special IFS build (“IC Gen build”) from a short run.
• Processing: split/modify GRIB, per-level parallelization, optional interpolation (Gaussian→spectral).
• Outputs: three IFS IC GRIB files (INIT/INIUA) + temp working directories.
• External dependencies: grib_filter, grib_set (ecCodes), parallel, optional cdo (<=2.4 for -C).
• generate_initial_conditions_parallel_only_IFS.sh: Orchestrator / pipeline
• runtime environment (HPC modules, filesystem paths)
• input/output GRIB artifacts
• processing stages inside the script
• external tools called by the script
Field modifications / metadata edits (grib_set)
5. Per-level parallel processing (GNU parallel, especially for 137 levels for upper-air)
Optional interpolation (cdo gp2sp etc. when -C)
• interpolate lnsp, vo, d, t from reduced Gaussian to spectral
7. Assembly/merge into final IC products (concatenate/merge GRIB messages into 3 output files)
Outputs (data store)
In <outdir>/IC_<idate>/:
• ICMSH<expid>INIT (spectral core fields)
• ICMGG<expid>INIT (surface grid-point fields)
• ICMGG<expid>INIUA (upper-air grid-point fields, all 137 levels)
Plus temps (explicitly show they are ephemeral):
• <outdir>/tmpdir/
• <outdir>/ictmpdir/
Downstream consumer (optional but useful)
• Next IFS-FESOM branched run consumes these three IC files.
Even though “FESOM ICs handled separately,” call that out as a boundary: this pipeline outputs IFS-atmos-only ICs.
Architectural patterns / design principles to highlight
This project’s “architecture” is basically:
• Batch ETL pipeline (Extract GRIB → Transform fields/metadata/interp → Load assembled GRIB)
• Embarrassingly parallel level processing (137 levels), implemented via GNU parallel
• Tool-based composition (thin orchestration script around stable CLI utilities)
• Strong file-contract interfaces (names/patterns like ICMGG<expid>+<step>)
• Restart file naming conventions and presence in <expdir>
• Exact tool availability/versions (esp. cdo constraint)
“Don’t overthink it” detail checklist (things to explicitly include)
• Exact input file triplet names and where they live (<expdir>)
• Exact output file triplet names and where they live (<outdir>/IC_<idate>/)
• Temp dirs recreated each run (tmpdir, ictmpdir)
• The optional interpolation path controlled by -C + CDO version caveat
• The parallelization unit: vertical levels (137) and that GNU parallel is required
• The boundary: “Atmosphere ICs only; FESOM ocean ICs are separate”