CODEWITHSUNDEEP
pythonpython-3.xmultiprocessingtypeerror
lte__
lte__

Reputation: 7576

Python TypeError - required bytes-like object instead of str

I'm trying to pass some JSON objects and two integers to a pool.

for i in range(0, multiprocessing.cpu_count()-1):
    fromindex = i * chunklen
    toindex = (i+1) * chunklen
    chunkedData.append([data['features'][fromindex:toindex], weekdaytopredict, hourtopredict])
chunkedData.append([data['features'][toindex:], weekdaytopredict, hourtopredict])
parallelstart = time.time()
result = (pool.map(parallelUpdateWithDT, chunkedData))

data is a geoJSON file with some polygons. I'd like to distribute these polygons for parallel processing. I pass n/cpu_count() polygons to the parallelUpdateWithDT function, which should further process them. My problem is a typeerror: Even though print(chunkedData) returns <class 'list'>, I geth the following error: TypeError: a bytes-like object is required, not 'str'. Where am I messing this up? Full stack trace:

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "<ipython-input-114-bf56cacb90b9>", line 34, in parallelUpdateWithDT
    if('rain' in result):
TypeError: a bytes-like object is required, not 'str'
"""

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
<ipython-input-115-031a5e24ee66> in <module>()
----> 1 decisionTreePrediciton(3, 5)

<ipython-input-114-bf56cacb90b9> in decisionTreePrediciton(weekdaytopredict, hourtopredict)
     15     print (type(chunkedData))
     16 
---> 17     result = (pool.map(parallelUpdateWithDT, chunkedData))
     18     parallelend = time.time()
     19 

/usr/lib/python3.5/multiprocessing/pool.py in map(self, func, iterable, chunksize)
    258         in a list that is returned.
    259         '''
--> 260         return self._map_async(func, iterable, mapstar, chunksize).get()
    261 
    262     def starmap(self, func, iterable, chunksize=None):

/usr/lib/python3.5/multiprocessing/pool.py in get(self, timeout)
    606             return self._value
    607         else:
--> 608             raise self._value
    609 
    610     def _set(self, i, obj):

A sample of chunkedData:

[[[{'geometry': {'coordinates': [[[10.914622377957983, 45.682007076150505], [10.927456267537572, 45.68179119797432], [10.927147329501077, 45.672795442796335], [10.914315493899755, 45.67301125363092], [10.914622377957983, 45.682007076150505]]], 'type': 'Polygon'}, ///////////////////////etc, waaay too big////////////, 'id': 6574, 'properties': {'cellId': 11454}}], 3, 5]

How is this an str? I don't get it. Thanks for any help!

Upvotes: 0

Views: 1021

Answers (1)

mgilson
mgilson

Reputation: 309969

It's impossible to tell from the code you posted, but I suspect that you are trying to check if a str is in a bytes. For example:

>>> bytes_obj = b'result'
>>> 'res' in bytes_obj
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: a bytes-like object is required, not 'str'

This implies that result in your code is of type bytes. There are two resolutions here. The first is to make 'rain' into a bytes object too:

if b'rain' in result:
    ...

The second is to turn result into a str:

result = result.decode(whatever_codec_it_should_be)

If you're going to take this second approach, you should convert result into a str at the earliest possible moment to avoid all sorts of str vs. bytes headaches. Usually, if you don't know you need a different codec, most things work off of utf-8 these days, so you could try that one...

Upvotes: 3

Related Questions