CODEWITHSUNDEEP
yamlyq
drekbour
drekbour

Reputation: 3081

Merge YAML into specific array entry with yq

Given master.yml ...

containers:
  - name: project-a
    items:
      - CCC
  - name: project-z
    items:
      - CCC
      - DDD

... and an update.yml ...

- name: project-z
  items:
    - CCC
    - EEE

... I want to merge it into the correct entry. This would give:

containers:
  - name: project-a
    items:
      - CCC
  - name: project-z
    items:
      - CCC
      - DDD
      - EEE

The following yq 4 works if the update was for project-a

yq ea 'select(fileIndex==0) * {"containers":select(fileIndex==1)}' master.yml update.yml

but, with the provided project-z update, it incorrectly replaces the first array entry (you end up with two project-zs).

Upvotes: 1

Views: 3166

Answers (2)

Mike Farah
Mike Farah

Reputation: 2564

This is similar to another stackoverflow question in which kev had a really neat solution for (I think).

I've since added his solution to yq docs here: https://mikefarah.gitbook.io/yq/operators/multiply-merge#merge-arrays-of-objects-together-matching-on-a-key

In your case, this will only work if the second file matched the structure of the first, (that is, it also has 'containers' as a parent):

yq eval-all '
(
  ((.containers[] | {.name: .}) as $item ireduce ({}; . *+ $item )) as $uniqueMap
  | ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
) as $mergedArray
| select(fi == 0) | .containers  = $mergedArray
` file.yaml file2.yaml

containers:
  - name: project-a
    items:
      - CCC
  - name: project-z
    items:
      - CCC
      - DDD
      - CCC
      - EEE

Basically it works by reducing into a merged map based on name (as you mention this would be much easier if that was already the case) and then converting that back to an array.

Disclaimer: I wrote yq

Upvotes: 3

drekbour
drekbour

Reputation: 3081

After a deep-dive into the manuals, I have this :

yq ea 'select(fi==1).0.name as $p | (select(fi==0).containers.[] | select(.name==$p)) = (select(fi==1) | .0 | . headComment="AUTO-UPDATED") |  select(fi==0)'  master.yml update.yml

which replaces rather than merges project-z by first searching name matching update.yml then completely replacing the content.

I understand the root cause being data is formatted as a list where it should be a dictionary (name is unique).

This would merge trivially and be better in future too!

containers:
  project-a:
    items:
      - CCC
  project-z:
    items:
      - CCC
      - DDD
      - EEE

Upvotes: 2

Related Questions