CODEWITHSUNDEEP
pythonpython-2.7python-3.xyamlprometheus
Suhas Chikkanna
Suhas Chikkanna

Reputation: 1520

Yaml : Found unexpected end of stream

I have written a python code to generate a YAML file(called targets.yml), which is read by a popular monitoring application called prometheus. The prometheus successfully reads the YAML file and its contents but it also throws an error in the log like below. 

level=error msg="Error reading file "/var/targets/targets.yml": yaml: 
line 218: found unexpected end of stream" source="file.go:199"

I am unable to get rid of this error, though I close the YAML file appropriately and below is the code for that:-

while True:

    create()
    with open('/var/targets/targets.yml', 'w') as output:
        print "opened the file to write"
        i=0
        for item in result:
            if(item != None and item['status'] == "ACTIVE"):
               print item['domains']['partner']
               print item['status']
               output.write("\n\n")
               output.write("- targets: ['" + "https://" + item["domains"]["agency"] + "']\n")
               output.write("  labels:\n")
               output.write("    alias: " + item["alias"])
               foo=item["name"]
           #print foo
               if isinstance(foo,basestring):
                  foo=foo.encode('utf8')
               else:
                  foo=unicode(foo).encode('utf8')
output.close()
print("Waiting for 300 seconds, before relooping")
time.sleep(100)

Also I do not think that my file extension makes any difference. Can somebody please suggest?

Upvotes: 1

Views: 28179

Answers (3)

Radu Gabriel
Radu Gabriel

Reputation: 3201

In my case there was a miss-match between open and close of quotes ", but the error was in another place then the line where it was reported by the error.

Upvotes: 4

Philip Stark
Philip Stark

Reputation: 638

EDIT: as brian-brazil has pointed out, this will not fix the problem, but it can possibly alleviate the symptoms. So only apply this answer if you absolutely cannot implement brian-brazil's answer.

To elaborate a bit on brian-brazil's answer: The fact that this operation is not atomic means that the other process (which knows nothing of your python script) can read your YAML file before it has finished writing.

If it is absolutely important that you write directly into that file, you might want to disable buffering (doc) and write your entire YAML document at once. You'd do that by collecting all the "write"s you're currently doing individually into a string and then writing all that at once.

instead of 

output.write("\n\n")
output.write("- targets: ['" + "https://" + item["domains"]["agency"] + "']\n")
output.write("  labels:\n")
output.write("    alias: " + item["alias"])

you'd do something like this:

yaml = ""
yaml += "\n\n"
yaml += "- targets: ['" + "https://" + item["domains"]["agency"] + "']\n"
yaml += "  labels:\n"
yaml += "    alias: " + item["alias"]
output.write(yaml)

This is not the most efficient way to do that, but for the sake of illustration, it should be fine.

Upvotes: 1

brian-brazil
brian-brazil

Reputation: 34152

with open('/var/targets/targets.yml', 'w') as output:

This is part of your problem, this isn't atomic. You need to create a temporary file and then move it into place.

I'd also recommend using a yaml library rather than creating it by hand.

Upvotes: 3

Related Questions