CODEWITHSUNDEEP
python-3.xdask
milind parikh
milind parikh

Reputation: 81

simple dask program not delaying functions on ubuntu 17.10, python 3.6.3, dask 1.1.1

Taken verbatim from the dask tutorial

from time import sleep

def inc(x):
    sleep(1)
    return x + 1

def add(x, y):
    sleep(1)
    return x + y
%%time
x = inc(1)
y = inc(2)
z = add(x, y)

CPU times: user 6.89 ms, sys: 628 µs, total: 7.51 ms Wall time: 3 s

from dask import delayed
%%time
x = delayed(inc)(1)
y = delayed(inc)(2)
z = delayed(add)(x, y)

CPU times: user 1.04 ms, sys: 97 µs, total: 1.13 ms Wall time: 1.23 ms

%%time
z.compute()

CPU times: user 3.78 ms, sys: 5.21 ms, total: 8.99 ms Wall time: 3.01 s

There is virtually no difference in wall time...whereas it should be less by about one second. What am I doing wrong?

Upvotes: 0

Views: 31

Answers (1)

milind parikh
milind parikh

Reputation: 81

Answering my own question....the key issue was that i was running a single core vm...so single thread...by running a distributed scheduler and two distributed workers (along a client ) made the expected parallelism work

Upvotes: 1

Related Questions