CODEWITHSUNDEEP
pythonnumpymemory-mapped-files
rowanwins
rowanwins

Reputation: 253

Load multiple memory mapped files with numpy

I'm trying to load two memory mapped files, 

temp = numpy.load(currentDirectory + "\\tmp\\temperature.npy", mmap_mode='r')
salinity = numpy.load(currentDirectory + "\\tmp\\salinity.npy", mmap_mode='r')

but Python throws the following error:

IOError: Failed to interpret file 'C:\\my\\file\\path\\..\\tmp\\salinity.npy' as a pickle

When I load either by itself, it works just fine.

The files that are quite large (~500MB), but otherwise I don't think they are notable.

What might the problem be here?

Upvotes: 0

Views: 420

Answers (1)

Mohbat Tharani
Mohbat Tharani

Reputation: 560

This works for me. Both files are large than 5GB. 

X = np.load(os.path.join(path, '_file1.npy'), mmap_mode='r')
Y = np.load(os.path.join(path, '_file2.npy'), mmap_mode='r')

Which operating system are you using? The problem is not with size of the "npy" files but problem is with "\" in path. change your path as:

path = '/media/gtx1060/DATA/Datasets'

Upvotes: 1

Related Questions