CODEWITHSUNDEEP
yamlyq
James Cockbain
James Cockbain

Reputation: 476

Merging YAML arrays, ignoring other fields in the YAML

I'm looking for a way to merge arrays from two separate YAMLs (appending one to the other). However the YAMLs have a field (not in the arrays) wrapped with {} to be replaced with runtime values. i.e

# yaml a
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  namespace: default
  name: {clusterRoleName}
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "watch", "list"]

# yaml b
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  namespace: default
  name: {clusterRoleName}
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "watch", "list"]

# desired
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  namespace: default
  name: {clusterRoleName}
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "watch", "list"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "watch", "list"]

I'm on yq version 2.14 for this. I've tried yq merge -a=append a.yaml b.yaml, which handles the rules arrays as I'd like but treats name: {clusterRoleName} as JSON and outputs:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: {ClusterRoleName: ''}
...

Is there a way to only merge one field, or ignore a specific key or type of value? I'm also not tied to using yq for this if someone is able to suggest an alternative method.

Upvotes: 0

Views: 471

Answers (1)

Raman Sailopal
Raman Sailopal

Reputation: 12867

While using a dedicated yaml parser such as yq would be ideal using awk maybe an alternative:

 awk 'NR==FNR && !/^[[:space:]]/ { tag=$1;next } tag=="rules:" && FNR==NR { map[idx++]=$0 } END { tag="" } NR!=FNR && !/^[[:space:]]/ { if (tag=="rules:") { for (i in map) { print map[i]}} tag=$1 } NR!=FNR { print }' yamlb yamla

Explanation:

awk 'NR==FNR && !/^[[:space:]]/ { # Processing yamlb (NR==FNR) and there there are spaces at the beginning of the line
         tag=$1; # Set the variable tag to the first space delimited field
         next # Skip to the next file
        } 
     tag=="rules:" && FNR==NR { # Process where tag is "rules:" and we are processing yamlb
          map[idx++]=$0 # Put the line in an array map with in incrementing index
        } 
     END { 
          tag="" # At the end of the file reset the variable tag
        } 
     NR!=FNR && !/^[[:space:]]/ { # Process yamla where there are no spaces at the start of the line
          if (tag=="rules:") { 
            for (i in map) { 
               print map[i] # If tag is equal to rules: print the array map
            }
          } 
          tag=$1 # Set the variable tag to the first space delimited field.
         } 
      NR!=FNR { 
          print # Print lines when we are processing yamla
         }' yamlb yamla

Upvotes: 1

Related Questions