CODEWITHSUNDEEP
linuxfindyamlconcatenationcat
NealR
NealR

Reputation: 10709

Concatenate multiple yaml files with seperator

I need to concat multiple k8s deployment yaml files into one deployment script, and in doing so create a specific separator --- between each file. I know the specific depth at which the files will live as well as the filename, however I don't know how many there will be at a given time, so I've used the find statement below to

  1. recursively search for the yaml file
  2. concat each
  3. piped in the tail command as a seperator

find . -type f -name 'deployment.yml' -exec cat {} + | tail -n +1 * > finalDeployment.yml

However, this creates broken yaml syntax by inserting the ==> <== delimeter:

enter image description here

I could simply have another task run a find/replace using the above as prefix/suffix tokens, however I'd like something more succinct within the above statement.

Is it possible to pipe in a specific character/set of characters a delimeter within a cat command, or is there another method to accomplish this?

Upvotes: 1

Views: 6680

Answers (1)

flyx
flyx

Reputation: 39738

What you want to do is not guaranteed to work. For example, you have these two YAML files:

foo: bar

and:

%YAML 1.2
---
baz

As you can see, the second file contains a directive. It also shows that --- in YAML is not a separator, but a directives end marker. It is optional if you don't have any directives like in the first document. If you concatenate both documents in the way you want to do it, you will get a document with two --- and %YAML 1.2 will be interpreted as content because it occurs after a directives end marker.

So what you actually want to do is to mark the end of each document with ..., the document end marker. After that marker, the parser is reset to its initial state, which guarantees that the second document is parsed exactly as it would have been when it was in a separate file.

Also, no harm is done by adding ... to the last document since it does not start another document implicitly. So your command could look like this (I depend on your statement that you know the depth at which the files lie here and as example, expect a depth of 3 directories):

echo -n > finalDeplayment.yml
for f in */*/*/deployment.yml; do
  cat $f >> finalDeployment.yml; echo "..." >> finalDeployment.yml
done

Upvotes: 1

Related Questions