HelmwaveCI

Read this in other languages: English, Русский

This is a core module for managing the entire company infrastructure using nested system, application, and project modules. Each module includes all dependent modules via Git submodules or symlinks and can be used independently of the parent.

• Quick Start
• Documentation
• DeepWiki

Motivation

• Standardization of infrastructure by eliminating the need to use "Docker Compose," including on developers' machines.
  • All typical tasks should be as simple to execute, such as mounting files and directories.
• The ability to completely eliminate duplication of common configuration parameters for any releases on any target clusters.
  • A single commit should describe the state of the entire infrastructure.
• The ability to reuse all configurations easily between related and unrelated projects and clusters.
• Enable developers to combine all their work on a single environment.

Key Features

• Dependency only on a single binary file and optionally some Makefiles.
• Implementation of an example fractal modular pattern for creating a personal monorepo using vanilla Git.
• Kubernetes (k8s) is all you need.
• Support for docker-compose modules

Example of Module Reuse

ci-all-my
  .ci-env # (optional Git submodule for secrets)
  ci # (Git submodule)
    <- possible deploy demo
  ci-all-work-1 # (Git submodule)
    .ci-env # (../.ci-env optional symlink for secrets)
    ci # (Git submodule)
    ci-lib-external-1 # (Git submodule)
    ci-lib-internal-1
    ci-project-external-1 # (Git submodule)
      ci # (Git submodule)
      ci-lib-external-1 # (Git submodule)
      <- possible deploy only this project (can be used by external clients)
    ci-project-internal-1
      ci # (../ci symlink)
      ci-lib-internal-1 # (../ci-lib-internal-1 symlink)
      ci-lib-external-1 # (../ci-lib-external-1 symlink)
      <- possible deploy only this project (for testing to prepare it for conversion to external)
    <- possible reuse of all work-1 projects
  ci-all-work-2 ...
  ci-my-hobby1 ...
  ci-* # (ci-all-*/ci* symlink)
  <- possible deploy all

File Structure of Each Module

• For Helmwave:
  • helmwave.yml.tpl - entry point with engine import
  • helmwave*.yml - these are the results of generating into release list
  • ci* - any sub modules (*/{src|charts})
  • charts/* - dumps from make chart_add_example
  • src/*.yml - entry points for different k8s contexts
    • src/lib/*.yml - context dependencies
    • src/dc/*/docker-compose.yml - compose modules
• bin - Makefiles
  • bin/watch

ci # core submodule with helmwave engine
  helmwave.yml.tpl # entry point with engine import
  charts # helm chart vendoring
  src
    _.yml # common context dependency forced to be connected
    _/**/<file> # static files for mounts
    chart/<chart>/<val>.tpl # values for chart releases
    lib
      <chart>/<val>.tpl # values for chart releases for all namespaces
      <dep>.yml # any context dependencies
    ns/<ns>/<chart>/<val>.tpl # values for chart releases in a specific namespace
    ctx/<ctx>/<ns>/<chart>/<val>.tpl # values for chart releases in a specific namespace and context
    <ctx>.yml # entry points for different k8s contexts
ci-project1/<ci>
ci-project2/<ci>
.env # for safe defaults HELMWAVE_*
.env.yml # common secret
.env-<ctx>.yml # context secret
helmwave.yml.tpl # entry point with engine import

k3d example

k3d cluster create local-ci -v "$PWD:$PWD@server:0" -p 80:80@loadbalancer --k3s-arg "--disable=traefik,local-storage,metrics-server@server:0" # 13.4s
helmwave up # 48.5s
curl http://localhost/api/openresty/
curl http://localhost/api/nginx/
curl http://localhost/nginx-raw/

helmwave up -t demo-wp # docker-compose example
open http://localhost

k3d cluster delete local-ci # 709ms

TODO:

• Add first sample module - infra.
• Add second sample module - mon.
• Docker-compose: take modules from namespace
• Docker-compose: ability to specify a node for services with disk mounting
• Docker-compose: add PVC
• Docker-compose: add file tree
• Docker-compose: pass external secrets in environment variables
• chart app: merge files and binaryData
• Add k3d automation.
• Add Kubespray automation.
• Add GitLab secrets management example.
• Add examples for many real work patterns.