CODEWITHSUNDEEP
pythonbash
pengz
pengz

Reputation: 2471

Yaml file for python and bash parsing

I have a simple list of hostnames such as:

hostname1,hostname2,hostname3

(Note: the format of the YAML file can be changed for easier parsing if necessary)

I need to be able to loop over this list of hostnames in both python and bash.

A simple indexed array will work for this purpose since the only property is hostname and the list will never be more than 1 level deep.

I know can easily parse this with Python with the code I started below.

How can I also parse this as a Bash array?

Python 2.7

import yaml
with open('emc_hosts.yaml', 'r') as f:
    doc = yaml.load(f)

doc = doc.split(",")

for v in doc:
    print(v)

UPDATE: The hostnames are all on the same line, but they don't have to be. I can create the hostname file in any format that I want including separated by return or other characters.

UPDATE 2: The file will only contain a list of hostnames.

UPDATE 3: As per a suggestion in the comments, I could easily change this from a YAML file to a simple text file with hostnames separated by new lines.

Upvotes: 1

Views: 2496

Answers (2)

Herve
Herve

Reputation: 127

You can parse yaml/json file directly in your shell/bash with niet.

Easy to install and easy to use:

$ pip install -U niet

Consider the following example:

$ cat dump.yaml
foo:
  bar:
    key: value
  baz:
    key: value
  tags:
    - one
    - two

You can parse this example file like this:

  $ niet dump.yaml foo.bar.key
  value
  $ for el in $(niet dump.yaml foo.tags); do echo ${el}; done
  one
  two

Niet have a good integration with shell and others bash like.

Niet yaml parser documentation, source code, and samples.

Also Niet is developing using python.

Upvotes: 3

Charles Duffy
Charles Duffy

Reputation: 295706

If you already have working Python code, might as well use it.

# Embed your Python script into your bash code
_get_hosts_python=$(cat <<'EOF'
import yaml, sys
with open(sys.argv[1], "r") as f:
    doc = yaml.load(f)

doc = doc.split(",")

for v in doc:
    print(v)
EOF
)

# provide a function to wrap its invocation
get_hosts() {
  IFS=$'\n' read -r -d '' -a "${1:-hosts}" \
    < <(python -c "$_get_hosts_python" "emc_hosts.yml" && printf '\0')
}

# and demonstrate how to actually use that function
get_hosts hostlist
echo "Got ${#hostlist[@]} hosts:"
printf '- %s\n' "${hostlist[@]}"

Note that bash 4.0 and newer have readarray and mapfile commands, which can be used to read a stream of lines into an array; I didn't use them here because it requires bash 4.4 to check the exit status of a process substitution (the <( ... ) syntax), so we wouldn't be able to detect errors if we went that approach (whereas here, the && printf \0' ensures that the stream has a trailing NUL -- and thus that the read has a successful exit status -- only if the Python code exited with a successful exit status).

Upvotes: 2

Related Questions