CODEWITHSUNDEEP
pythonyaml
Mike
Mike

Reputation: 7841

python convert to yaml formatting issue

I have the following simple script

#!/usr/bin/env python

import yaml    

returns = {'file_|-/usr/bin/gen-motd.py_|-/usr/bin/gen-motd.py_|-managed': {'comment': 'File /usr/bin/gen-motd.py updated', 'pchanges': {'diff': '--- \n+++ \n@@ -1,5 +1,7 @@\n #!/usr/bin/env python\n \n+import sys\n+import urllib2\n import socket\n import json\n import time\n'}, 'name': '/usr/bin/gen-motd.py', 'start_time': '16:18:54.060168', 'result': True, 'duration': 99.663, '__run_num__': 1, 'changes': {'diff': '--- \n+++ \n@@ -1,5 +1,7 @@\n #!/usr/bin/env python\n \n+import sys\n+import urllib2\n import socket\n import json\n import time\n'}, '__id__': '/usr/bin/gen-motd.py'}}
returns = yaml.dump(returns, default_flow_style=False)
print returns

Its outputting

file_|-/usr/bin/gen-motd.py_|-/usr/bin/gen-motd.py_|-managed:
  __id__: /usr/bin/gen-motd.py
  __run_num__: 1
  changes:
    diff: "--- \n+++ \n@@ -1,5 +1,7 @@\n #!/usr/bin/env python\n \n+import sys\n+import\
      \ urllib2\n import socket\n import json\n import time\n"
  comment: File /usr/bin/gen-motd.py updated
  duration: 99.663
  name: /usr/bin/gen-motd.py
  pchanges:
    diff: "--- \n+++ \n@@ -1,5 +1,7 @@\n #!/usr/bin/env python\n \n+import sys\n+import\
      \ urllib2\n import socket\n import json\n import time\n"
  result: true
  start_time: '16:18:54.060168'

The diff part is badly formatted.. I want it to return something like

file_|-/usr/bin/gen-motd.py_|-/usr/bin/gen-motd.py_|-managed: 
  __id__: /usr/bin/gen-motd.py
  __run_num__: 1
  changes: 
    diff: |
        --- 
        +++ 
        @@ -1,5 +1,7 @@
         #!/usr/bin/env python

        +import sys
        +import urllib2
         import socket
         import json
         import time
  comment: "File /usr/bin/gen-motd.py updated"
  duration: 99.663
  name: /usr/bin/gen-motd.py
  pchanges: 
    diff: |
        --- 
        +++ 
        @@ -1,5 +1,7 @@
         #!/usr/bin/env python

        +import sys
        +import urllib2
         import socket
         import json
         import time
  result: true
  start_time: "16:18:54.060168"

Upvotes: 1

Views: 249

Answers (1)

Anthon
Anthon

Reputation: 76632

There are two problems to overcome to get this as you want:

  • your lines cannot have spaces before the newline, as there is no way to represent that in literal style scalars
  • you need to provide a routine that outputs strings with newlines as literal style scalars or use a YAML library that supports that format, like ruamel.yaml (disclaimer: I am the author of that package).

Using ruamel.yaml is the far more easy way, just convert strings that are 3 levels deep and contain a newline:

import sys
import ruamel.yaml
from ruamel.yaml.scalarstring import LiteralScalarString

returns = {'file_|-/usr/bin/gen-motd.py_|-/usr/bin/gen-motd.py_|-managed': {'comment': 'File /usr/bin/gen-motd.py updated', 'pchanges': {'diff': '--- \n+++ \n@@ -1,5 +1,7 @@\n #!/usr/bin/env python\n \n+import sys\n+import urllib2\n import socket\n import json\n import time\n'}, 'name': '/usr/bin/gen-motd.py', 'start_time': '16:18:54.060168', 'result': True, 'duration': 99.663, '__run_num__': 1, 'changes': {'diff': '--- \n+++ \n@@ -1,5 +1,7 @@\n #!/usr/bin/env python\n \n+import sys\n+import urllib2\n import socket\n import json\n import time\n'}, '__id__': '/usr/bin/gen-motd.py'}}

for k in returns:
    for k1 in returns[k]:
        if not isinstance(returns[k][k1], dict):
            continue
        for k2 in returns[k][k1]:
            v = returns[k][k1][k2]
            try:
                if '\n' in v:
                    while ' \n' in v:
                        v = v.replace(' \n', '\n') # remove EOL spaces
                    returns[k][k1][k2] = LiteralScalarString(v)
            except TypeError:
                continue

yaml = ruamel.yaml.YAML()
yaml.dump(returns, sys.stdout)

You can of course remove the whole for construction and use 

returns = {'file_|-/usr/bin/gen-motd.py_|-/usr/bin/gen-motd.py_|-managed': {'comment': 'File /usr/bin/gen-motd.py updated', 'pchanges': {'diff': PreservedScalarString('---\n+++\n@@ -1,5 +1,7 @@\n #!/usr/bin/env python\n\n+import sys\n+import urllib2\n import socket\n import json\n import time\n')}, 'name': '/usr/bin/gen-motd.py', 'start_time': '16:18:54.060168', 'result': True, 'duration': 99.663, '__run_num__': 1, 'changes': {'diff': PreservedScalarString('---\n+++\n@@ -1,5 +1,7 @@\n #!/usr/bin/env python\n\n+import sys\n+import urllib2\n import socket\n import json\n import time\n')}, '__id__': '/usr/bin/gen-motd.py'}}

Either way this gives you:

file_|-/usr/bin/gen-motd.py_|-/usr/bin/gen-motd.py_|-managed:
  start_time: '16:18:54.060168'
  comment: File /usr/bin/gen-motd.py updated
  duration: 99.663
  __id__: /usr/bin/gen-motd.py
  changes:
    diff: |
      ---
      +++
      @@ -1,5 +1,7 @@
       #!/usr/bin/env python

      +import sys
      +import urllib2
       import socket
       import json
       import time
  __run_num__: 1
  pchanges:
    diff: |
      ---
      +++
      @@ -1,5 +1,7 @@
       #!/usr/bin/env python

      +import sys
      +import urllib2
       import socket
       import json
       import time
  name: /usr/bin/gen-motd.py
  result: true

Because you are using dict the order of the keys in the YAML file is undetermined. You can construct returns directly with CommentedMap instances (from ruamel.yaml.comments) and PreservedScalarString and have control over the key orderings in your YAML mappings as well.

Upvotes: 2

Related Questions