How do you apply a recursive formatting with Go Templates (Helm)?

Question

I'm using helm and given a yaml object I want to flatten it while applying some recursive formatting.

Given this:

some_map:
  with: different
  indentation:
    levels: and
  nested:
    sub: 
      maps: "42"
    and_more:
      maps: 42

I want to (for example) get this:

some_map.with="different"
some_map.indentation.levels="and"
some_map.nested.sub.maps="42"
some_map.nested.and_more.maps=42

I haven't read anything about recursive looping in the helm docs, keep in mind that the format of the recursion in the example ( "%v.%v" if !root else "%v=%v" ) may vary.

Answer 1

another approach

_helpers.tpl:

{{- define "template.flattenFn" -}}
  {{- $ctx := . -}}
  {{- if or (eq (kindOf .data) "map") (eq (kindOf .data) "slice") }}
    {{- range $key, $value := .data }}
      {{- include "template.flattenFn" (dict "prefixes" (append $ctx.prefixes $key) "data" $value ) }}
    {{- end }}
  {{- else if .prefixes }}
    {{- printf "\"%s\":%s,"  (join "__" .prefixes) ( mustToJson ( .data | toString )) }}
  {{- end }}
{{- end -}}

{{- define "template.flatten" -}}
{{- $result := include "template.flattenFn" (dict "prefixes" list "data" . ) -}}
{ {{- trimSuffix "," $result -}} }
{{- end -}}

values.yaml

  environment_map:
    Log:
      Level:
        Default: Debug
      Target:
      - Name: Console
        Args:
          formatter: "JsonFormatter"

usage

{{- range $env, $value := include "template.flatten" .Values.environment_map | fromYaml }}
  - name: {{ $env }}
    value: {{ $value | quote }}
{{- end }}

result

    - name: Log__Level__Default
      value: "Debug"
    - name: Log__Target__0__Args__formatter
      value: "JsonFormatter"
    - name: Log__Target__0__Name
      value: "Console"

Answer 2

Thanks to @mdaniel for his answer, which helped/allowed me to fix my problem! His (quick, I get it) solution has problem though if you have move values with same prefix. To carry on with his example:

{{ $fred := dict
    "alpha" (dict "a0" "a0ch0")
    "beta" (dict "beta0" (dict "beta00" 1234))
    "charlie" (list "ch0" "ch1" "ch2")
    "problem" (dict "beta0" (dict "1" "1" "2" "2" )) }}
data:
  theData: |
{{ toJson $fred | indent 4 }}
  toml: |
{{ include "bob" $fred | indent 4 }}

will produce:

data:
  theData: |
    {"alpha":{"a0":"a0ch0"},"beta":{"beta0":{"beta00":1234}},"charlie":["ch0","ch1","ch2"],"problem":{"beta0":{"1":"1","2":"2"}}}
  toml: |
    problem.beta0.1="1"
    2="2"
    alpha.a0="a0ch0"
    beta.beta0.beta00=1234
    charlie=["ch0","ch1","ch2"]

which is not correct. Also there is another eye-candy problem of lost order, no idea why or how to fix it, but that's not such biggie.

My extra requirement was, that I needed to add common prefix to all lines. Without it you can define another template simplifying usage, but that's trivial to do.

disclaimer: I'm totally new to helm/go-templating, so following isn't probably optimal, but it should fix these issues.

To carry on with example again:

{{ $fred := dict
    "alpha" (dict "a0" "a0ch0")
    "beta" (dict "beta0" (dict "beta00" 1234))
    "charlie" (list "ch0" "ch1" "ch2")
    "problem" (dict "beta0" (dict "1" "1" "2" "2" )) }}
data:
  theData: |
{{ toJson $fred | indent 4 }}
  toml: |
{{ include "flattenYaml" (dict "prefix" "added_prefix" "data" $fred) | indent 4 }}

will produce:

data:
  theData: |
    {"alpha":{"a0":"a0ch0"},"beta":{"beta0":{"beta00":1234}},"charlie":["ch0","ch1","ch2"],"problem":{"beta0":{"1":"1","2":"2"}}}
  toml: |
    added_prefix.alpha.a0="a0ch0"
    added_prefix.beta.beta0.beta00=1234
    added_prefix.charlie=["ch0","ch1","ch2"]
    added_prefix.problem.beta0.1="1"
    added_prefix.problem.beta0.2="2"

and template source for flattenYaml looks like this:

{{- define "flattenYaml" -}}
{{- $dict := . -}}
{{- $prefix := $dict.prefix -}}
{{- $data := $dict.data -}}
{{- $knd := kindOf $data -}}
    {{- if eq $knd "map" }}
        {{- range (keys $data) }}
            {{- $key := . }}
            {{- $prefixedKey := (printf "%s.%s" $prefix $key) }}
            {{- $value := get $data $key }}
            {{- $valueKind := kindOf $value }}
            {{- if eq $valueKind "map" }}
                {{-   include "flattenYaml" (dict "prefix" ($prefixedKey) "data" $value) }}
            {{- else }}
                {{-   printf "%s=%s\n" $prefixedKey (toJson $value) }}
            {{- end }}
        {{- end }}
    {{- else }}
        {{ toJson . }}#k({{ $knd }})
    {{- end }}
{{- end -}}

disclaimer 2: I have no idea what #k ... in {{ toJson . }}#k({{ $knd }}) from original solution does, do not be surprised if it explodes into your face ;)

edit: the

    {{ toJson . }}#k({{ $knd }})

actually blew into my face, still don't know what #k should serve, but following replacement fixed my issue:

    {{- if ne $data nil }}
        {{- toJson $data }}
    {{- end }}

Answer 3

Yes, it seems that {{ define supports recursive use of {{ include, although unknown to what depth

The PoC I whipped up to see if it could work

{{- define "bob" -}}
{{- $it := . -}}
{{- $knd := kindOf . -}}
{{- if eq $knd "map" }}
{{- range (keys .) }}
{{- $k := . }}
{{- $v := get $it . }}
{{- $vk := kindOf $v }}
{{- if eq $vk "map" }}
{{-   printf "%s." $k }}
{{-   include "bob" $v }}
{{- else }}
{{-   printf "%s=%s\n" $k (toJson $v) }}
{{- end }}
{{- end }}
{{- else }}
{{ toJson . }}#k({{ $knd }})
{{- end }}
{{- end -}}

invoked as

{{ $fred := dict 
    "alpha" (dict "a0" "a0ch0")
    "beta" (dict "beta0" (dict "beta00" 1234))
    "charlie" (list "ch0" "ch1" "ch2") }}
data:
  theData: |
{{ toJson $fred | indent 4 }}
  toml: |
{{ include "bob" $fred | indent 4 }}

produced

data:
  theData: |
    {"alpha":{"a0":"a0ch0"},"beta":{"beta0":{"beta00":1234}},"charlie":["ch0","ch1","ch2"]}
  toml: |
    alpha.a0="a0ch0"
    beta.beta0.beta00=1234
    charlie=["ch0","ch1","ch2"]

Also, your cited example seems to make reference to the outermost variable name, which I don't think helm knows about, so you'd need an artificial wrapper dict in order to get that behavior: {{ include "toToml" (dict "some_map" .Values.some_map) }}