How netcracker/qubership-envgene works
Project type and purpose
• This is a command-line tool and library suite (“EnvGene”) for generating cloud/on-prem environments from versioned templates and inventories, using Python, Ansible, JSON/YAML schemas, and Git.
• It ships as multiple sub-modules (build_effective_set_generator, build_envgene, build_gcip, env-builder, plus Python packages and scripts), each containerized via Docker and orchestrated in CI workflows.
File structure
• Top-level modules:
– build_effective_set_generator (Ansible roles & Python scripts to produce “effective parameter sets”)
– build_envgene (Ansible roles & scripts to scaffold and deploy envs)
– build_gcip (Python pipelines/plugins for Google CIP jobs)
– env-builder (Ansible playbooks/roles for initial environment descriptor scaffolding)
– python/envgene (core CLI & helper library)
– python/integration & python/jschon-sort (supporting Python packages)
– schemas (JSON schemas for validation)
– docs, samples, scripts, tests, CI workflows (GitHub Actions, Dockerfiles)
• Patterns:
– Plugin-based core (python/envgenehelper + plugin_engine)
– Ansible-driven declarative execution
– Dockerized builds for distribution and CI
– Git as the data store for versioned templates, credentials (encrypted), inventories.
README for technical stack
• Core languages: Python 3.x, YAML/JSON, Jinja2 templates
• Execution engines: Ansible, custom Python CLI, plugin engine
• External systems: Git (diffs, merges, history), Docker (packaging), GitHub Actions (CI/CD).
a. CLI & Core Library
• python/envgene: the entry point (__main__.py), uses envgenehelper modules (config, crypto, schema validation, file & JSON/YAML helpers, merge helpers, deployer).
• plugin_engine within build_gcip: dynamic loading of per-env plugins and use-case adapters.
b. Ansible Execution Engine
• Each “build_…” module defines its own ansible.cfg, filter_plugins, library plugins, playbooks, and roles.
• Env-builder module scaffolds base descriptors from templates.
c. Templates & Schemas
• Jinja2 templates under samples/templates & env-builder/templates.
• JSON/YAML schema files under schemas/ for validating descriptors, parameters, profiles.
d. Data Repositories
• Git repos containing templates, inventories, credentials (using SOPS/fernet encryption).
• Local file system I/O.
e. CI/CD & Containerization
• Dockerfiles under each build_… module for packaging.
• GitHub Actions workflows under .github/workflows and github_workflows/instance-repo-pipeline.
Relationships & data flows
• User invokes envgene CLI → reads config / inventory / credentials / template definitions → runs schema validation → merges parameters → spawns Ansible playbooks or internal Python ‘deployer’ → outputs generated environment descriptors or applies changes.
• Effective-set generator: takes current parameter state + mapping schema → outputs consolidated parameter set.
• build_gcip: coordinates CI pipeline definitions for GitLab/GitHub, using plugin_engine to tailor per-cloud-infra jobs.
• env-builder: scaffolds blank “Solution / Tenant / Namespace” structures from Jinja templates.
• All operations versioned in Git; encryption backends secure credentials in-repo.
Architectural patterns & design principles
• Modular, plugin-centric design: each subproject is independent, shares common helpers in python/envgenehelper.
• Declarative configuration via Ansible + Jinja2 templates.
• Schema-driven validation ensures correctness before generation.
• Dockerized components ensure reproducible builds in CI/CD pipelines.
• Grouping of modules that share the python/envgenehelper library.