CODEWITHSUNDEEP
python-3.xpython-2.7daskdask-distributeddask-delayed
TheCodeCache
TheCodeCache

Reputation: 972

distributed.protocol.pickle - INFO - Failed to serialize. Exception: Pickling an AuthenticationString object is disallowed for security reasons

python code::

from dask.distributed import variable, Client
from multiprocessing import Process, current_process

def my_task(proc):
    print("process object::", proc)

def doubler(number):
    # do stuff returns something

proc = Process(target=doubler, args=(numbers,)) # creating a python process
client = Client()
future = client.submit(my_task, proc)

Upon running the above code, I am getting serialization error for proc object. please help me to figure out the issue here..

distributed.protocol.pickle - INFO - Failed to serialize (<Process(Process-1, initial)>,). Exception: Pickling an AuthenticationString object is disallowed for security reasons
Traceback (most recent call last):
  File "stop_task.py", line 42, in <module>
    future = client.submit(my_task, proc)
  File "/usr/lib/python2.7/site-packages/distributed/client.py", line 1121, in submit
    retries={skey: retries} if retries else None)
  File "/usr/lib/python2.7/site-packages/distributed/client.py", line 1954, in _graph_to_futures
    'tasks': valmap(dumps_task, dsk3),
  File "/usr/lib/python2.7/site-packages/toolz/dicttoolz.py", line 84, in valmap
    rv.update(zip(iterkeys(d), map(func, itervalues(d))))
  File "/usr/lib/python2.7/site-packages/distributed/worker.py", line 703, in dumps_task
    'args': pickle.dumps(task[1:])}
  File "/usr/lib/python2.7/site-packages/distributed/protocol/pickle.py", line 51, in dumps
    return cloudpickle.dumps(x, protocol=pickle.HIGHEST_PROTOCOL)
  File "/usr/lib/python2.7/site-packages/cloudpickle/cloudpickle.py", line 881, in dumps
    cp.dump(obj)
  File "/usr/lib/python2.7/site-packages/cloudpickle/cloudpickle.py", line 268, in dump
    return Pickler.dump(self, obj)
  File "/usr/lib64/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib64/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.7/pickle.py", line 548, in save_tuple
    save(element)
  File "/usr/lib64/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib64/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib64/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib64/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib64/python2.7/pickle.py", line 681, in _batch_setitems
    save(v)
  File "/usr/lib64/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 290, in __reduce__
    'Pickling an AuthenticationString object is '
TypeError: Pickling an AuthenticationString object is disallowed for security reasons

not sure what is going wrong here.. Appreciate your time and help.. Thanks..

Upvotes: 2

Views: 1595

Answers (1)

MRocklin
MRocklin

Reputation: 57281

It looks like Process objects are not serializable. This is not surprising. A Process on one machine does not translate well to processes on other machines. This sort of object is too hard to move around, so it fails to serialize.

Upvotes: 1

Related Questions