helm filter over range

Question

Is it possible to filter the results of a range in helm? For example the values file contains a list of maps like:

clients:
  - name: clientA
    id: id001
    user: usernameA
    pass: passwordA
  - name: clientB
    id: id002
    user: usernameB
    pass: passwordB
  - name: clientA
    id: id003
    user: usernameA
    pass: passwordA

In my template I need to extract only the unique values of user and pass. I have something like this:

{{- range .Values.clients }}
- name: {{ .name | printf "user_%s" }}
  valueFrom:
    secretKeyRef:
      name: somesecret
      key: {{ .user | quote }}
- name: {{ .name | printf "pass_%s" }}
  valueFrom:
    secretKeyRef:
      name: somesecret
      key: {{ .pass | quote }}
{{- end }}  

The end result should be something like:

- name: user_clientA
  valueFrom:
    secretKeyRef:
      name: somesecret
      key: usernameA
- name: pass_clientA
  valueFrom:
    secretKeyRef:
      name: somesecret
      key: passwordA
- name: user_clientB
  valueFrom:
    secretKeyRef:
      name: somesecret
      key: usernameB
- name: pass_clientB
  valueFrom:
    secretKeyRef:
      name: somesecret
      key: passwordB

I tried {{- range .Values.clients | pick "user" "pass" | uniq }} but doesn't seem to work. I don't know how to write this so i can filter just "user" and "pass" keys and discard the duplicates.

Answer 1

Even with its extensions, Helm doesn't support deep filtering of lists like you propose. There's not a great way to say "filter this list to the unique values of this map item", or "given an item, is it the first one in the list with some property". In a more functional language I could imagine using constructs like map or filter here, and Helm has no equivalents to these.

In practice the thing I'd be most likely to do is to declare this an invalid configuration. It would be lazy, but not unreasonable, to just execute the template as you've shown it. (If this is in a pod spec's env: block, I believe this would produce a valid object and the last setting of each environment variable takes effect.)

It's possible to do this checking in Helm templates; but it's a lot of code, in an unfamiliar language, that's hard to test. Also consider whether something like an operator could be a better implementation choice for you, since this can be written in a more standard language.

---

If you believe the length of the list will be reasonably short, you can write a recursive template to process this. It would receive the list of remaining items and the list of names it's already seen. If the list of items is empty, stop; if the current item has been seen, call ourselves with the rest of the list and the same seen list; otherwise emit the item, add it to the "seen" list, and call ourselves. In Python pseudocode this could look like:

def helper(remaining, seen):
  if len(remaining) == 0:
    # do nothing if the list is empty
    return
  else:
    first = remaining[0]
    name = first['name']
    rest = remaining[1:]
    if name in seen:
      # we have already seen this item; skip it and go to the next one
      return helper(rest, seen)
    else:
      # this is a new item; emit it and remember its name
      emit(first)
      new_seen = seen + [name]
      return helper(rest, new_seen)

def process(the_list):
  helper(the_list, [])

You can translate that logic to Helm templates, with the additional trick of packing the multiple parameters into a single list, since the Go text/template templates only take a single parameter. (If you need the top-level Helm object for .Values, .Release, .Files, etc. you need to pass that explicitly as well.)

{{- define "helper" -}}
  {{- $remaining = index . 0 -}}
  {{- $seen := index . 1 -}}
  {{- if empty $remaining -}}
    {{-/* do nothing */-}}
  {{- else -}}
    {{- $first := first $remaining -}}
    {{- $name := $first.name -}}
    {{- $rest := rest $remaining -}}
    {{- if has $name $seen -}}
      {{-/* recurse without emitting anything */-}}
      {{- template "helper" (list $rest $seen) -}}
    {{- else -}}
      {{-/* emit the item */-}}
- name: user_{{ $first.name  }}
  valueFrom:
    secretKeyRef:
      name: somesecret
      key: {{ quote $first.user }}
- name: pass_{{ $first.name }}
  valueFrom:
    secretKeyRef:
      name: somesecret
      key: {{ quote $first.pass }}
{{/* ...then move on to the next item */-}}
      {{- $new_seen := append $seen $first.name -}}
      {{- template "helper" (list $first $new_seen) -}}
    {{- end -}}
  {{- end -}}
{{- end -}}

env:
{{ include "helper" (list .Values.clients list) | indent 2 }}