Looping through Gitlab-ci inputs array type

Question

With introduction of the array type for Gitlab-ci inputs I have wanted to do things such as looping through the contents of that array, however what I have found is that even though it's set as a type of array, bash does not see it as a conventional array in bash.

For example if we have something like this:

spec:
  inputs:
    tags:
      type: array
      default:
         - "${CI_COMMIT_SHA}"
         - "latest"

---

print-tags:
   image: alpine:latest
   stage: build
   script: |
       tags=$[[ inputs.tags]]
       for tag in "${tags[@]}"    
       do
          echo $tag
       done

It will produce an error like: /busybox/sh: eval: line 186: latest]: not found.

So how can we loop through inputs declared with the type of array?

Answer 1

I recently came across this problem and was able to come up with a solution that should cater to all possibilities of array inputs.

What I mean by that is GitLab CI input arrays are a bit odd. They're a little like JSON arrays, except they allow values that aren't wrapped in quotes as well as those unquoted values having spaces. For example, the following is totally valid:

my_array_values:
  - value_1
  - "value 2"
  - value 3
  - "value 4"

This will cause $[[ inputs.my_array_values ]] to hold the following value:

'[value_1, "value 2", value 3, "value 4"]'

Hence, we can't use jq since it isn't valid JSON. You could enforce the use of "" wrapped values within your team / organisation and then use jq. However, if you want a more robust solution that strips quotes off quoted values you could do:

script:
  -|
   readarray -t values < <(echo $[[ inputs.my_array_values ]] |
     sed '
       s/[][]//g; # remove brackets
       s/,/\n/g; # remove commas
       s/\"//g; # remove double quotes
     '
   for value in ${values[@]}; do
     echo "$value"
   done

Now, if you wanted to also modify each element by adding a prefix (which I needed to do) you'll also need to strip whitespace after commas and then escape any remaining spaces:

script:
  -|
   readarray -t values < <(echo $[[ inputs.my_array_values ]] |
     sed '
       s/[][]//g; # remove brackets
       s/,\s\+/,/g; # remove spaces after commas
       s/,/\n/g; # remove commas
       s/\"//g; # remove double quotes
       s/ /\\ /g; # escape spaces
     ' |
     xargs printf "prefix/%s\n")
   )
   for value in ${values[@]}; do
     echo "$value"
   done

If anyone that's good at bash has a better way of doing this please let me know

Answer 2

In our environment, the rendered input also appeared as ["foo", "bar", "baz"], which is actually the JSON representation of the input array. Attempting to iterate over this input with a for loop does not work because it is a single JSON-formatted string rather than a Bash array.

The suggested approach to use tr to remove special characters like " and , wasn't a suitable approach for us, as it could inadvertently remove characters we wanted to keep in the input. Instead, we opted to use jq to parse the JSON string, allowing us to process each element in the array individually, preserving any contained commas, quotes etc.:

echo '$[[ inputs.set_env_vars ]]' | jq -r '.[]' | while read -r entry; do
  # do something with $entry
done

Answer 3

The answer given Popeye above almost worked for me.

In my case, the array looked like this: ["ed64f4cc", "latest"]. That is, I had to remove the extra , in there. I modified the command to tags=$(echo "$[[ inputs.tags ]]" | tr -d '[].",') (note the extra , in the tr command). Finally, I had to remove the "" around "${tags}". The final solution that worked for me was:

spec:
  inputs:
    tags:
      type: array
      default:
         - "${CI_COMMIT_SHA}"
         - "latest"

---

print-tags:
   image: alpine:latest
   stage: build
   script: |
       tags=$(echo "$[[ inputs.tags ]]" | tr -d '[].",')
       for tag in ${tags}    
       do
          echo $tag
       done

Answer 4

Answering my own question as I spent a while on this problem a couple weeks back and struggled to find anything.

If we take a look at what $[[ inputs.tags ]] actually prints out to console it would look something like: [ed64f4cc, latest], but when we loop through using for tag in "${tags[@]}" it seems to include the [ and ] as part of each entry in the so called array. I think it's not an array in the conventional sense that bash likes it in and it's actually converted to a String and the for loop is treating the , as the delimiter and sees the [ and ] as being part of each value in that string.

What I have found that works for me is removing the [ and ] beforehand like: tags=$(echo "$[[ inputs.tags ]]" | tr -d '[]."') which allows me to loop through using for tag in ${tags}

Full example:

spec:
  inputs:
    tags:
      type: array
      default:
         - "${CI_COMMIT_SHA}"
         - "latest"

---

print-tags:
   image: alpine:latest
   stage: build
   script: |
       tags=$(echo "$[[ inputs.tags ]]" | tr -d '[]."')
       for tag in "${tags}"    
       do
          echo $tag
       done

Which then gives me the output of:

ed64f4cc

latest