Reputation: 53
I am receiving a strange error when trying to turn a bytes object into a string:
Traceback (most recent call last):
File "/home/Scraper_v1.0.py", line 149, in <module>
rawjsonstr = str(rawjsonstr, 'utf-16')
TypeError: decoding str is not supported
My code is as follows:
# preparing string
try:
rawjsonstr = "".join(rawjson[3]).encode('utf-16') # used to select the correct text/javascript node
except IndexError:
errorcount += 1
with open(filepath, "at") as f:
write = csv.writer(f)
write.writerow(["Error: no valid JSON found",
statenum,
statesubnum,
suburbnum,
listingnum,
listingsurlstr])
pass
rmvlist = ["var digitalData = ",
"var titanEnabled = true;",
"titan = titan || {};",
"// enable the command queue to allow spa execution in the correct order",
"titan.cmd = titan.cmd || [];"]
rawjsonstr = str(rawjsonstr, 'utf-16')
Edit
Re-ran the above code without encoding/decoding. Now receive the following error on the same loop.
Traceback (most recent call last):
File "/home/isaac/PycharmProjects/ResidentialData/AllHomesScraper_v1.0.py", line 162, in <module>
jsondata = json.loads(rawjsonstr)
File "/usr/lib64/python3.4/json/__init__.py", line 318, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.4/json/decoder.py", line 343, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.4/json/decoder.py", line 359, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting ',' delimiter: line 1 column 1033 (char 1032)
I will re-run it and capture the string that is causing this failure (looks like it's just incorrectly formatted JSON data).
Upvotes: 1
Views: 11586
Reputation: 53
The error was being caused as a result of the program trying to parse an invalid JSON string: JSON formatter
Solution: Catch the error and append a '{' to the end of the string when it occurs.
Upvotes: 1