CODEWITHSUNDEEP
pythonjoblib
Him
Him

Reputation: 5549

joblib dump writes 0 byte file

joblib.dump doesn't seem to do anything for me whatsoever. Possibly I'm having some versioning conflicts or something. Any help is appreciated.

My joblib version: 0.13.2
Seems to affect 0.14.0 as well

To reproduce:

import joblib
import os

foo = open("bar", "w+b")
joblib.dump("test", foo)
print(os.stat("bar").st_size)

#prints 0... expect the size of a file containing the pickled string "test" > 0 bytes

Upvotes: 1

Views: 291

Answers (1)

nicoring
nicoring

Reputation: 683

The buffer has not been written to disk, so the file has been created but has no content. You have to flush the internal buffer then the content is written into the file:

>>> foo = open("bar", "w+b")
>>> joblib.dump("test", foo)
>>> foo.flush()
>>> print(os.stat("bar").st_size)
14

Or use a context manager which does that automatically:

>>> with open("bar", "w+b") as foo:
>>>     joblib.dump("test", foo)
>>> print(os.stat("bar").st_size)
14

Or you can disable buffering:

>>> foo = open("bar", "w+b", buffering=0)
>>> joblib.dump("test", foo)
>>> print(os.stat("bar").st_size)
14

Also have a look here maybe: How often does python flush to a file?

Upvotes: 1

Related Questions