CODEWITHSUNDEEP
pythonyaml
daydreamer
daydreamer

Reputation: 91949

YAML - substituting values for variables?

consider the following yaml 

hadoop:  
   storage: '/x/y/z/a/b'
   streaming_jar_path: '/x/c/d/f/r/*.jar'
   commands:  
       mkdir: 'hadoop dfs -mkdir dir'
       copyFromLocal: 'hadoop dfs -copyFromLocal from_path to_path'  
       run: 'hadoop jar $streaming_jar_path -mapper mapper_path -reducer reducer_path -input hdfs_input -output hdfs_output'

I want to substitute the value of streaming_jar_path to $streaming_jar_path, how can I do that?

I know we can merge the hashes using &(anchors) but here i just want to change one value
I am sorry if this is trivial thing, I am very new to YAML

Thank you

Upvotes: 5

Views: 9745

Answers (2)

Akif
Akif

Reputation: 6776

You can restructure your YAML file and execute with Ansible.

commands.yml:

- hosts: localhost
  vars:
    streaming_jar_path: '/x/c/d/f/r/*.jar'
  tasks:
    - name: mkdir
      shell: "hadoop dfs -mkdir dir"
    - name: copyFromLocal
      shell: "hadoop dfs -copyFromLocal from_path to_path"
    - name: run
      shell: "hadoop jar {{ streaming_jar_path }} -mapper mapper_path -reducer reducer_path -input hdfs_input -output hdfs_output"

Then simply run ansible-playbook to execute shell commands:

ansible-playbook commands.yml

Upvotes: 5

Graeme Stuart
Graeme Stuart

Reputation: 6053

This should be a simple process of reading your file, editing the data and writing back to file.

import yaml

infile = 'input.yaml'
outfile = 'output.yaml'

#read raw yaml data from file into dict
with open(infile, 'r') as f:
    data = yaml.load(f.read())

#make changes to dict
data['hadoop']['streaming_jar_path'] = '$streaming_jar_path'

#write dict back to yaml file
with open(outfile, 'w') as f:
    f.write(yaml.dump(data))

Upvotes: 0

Related Questions