CODEWITHSUNDEEP
pythonnumpysaveloadbytesio
Rikard Olsson
Rikard Olsson

Reputation: 869

Load numpy arrays from compressed bytes

I'm trying to save (using savez_compressed) a bunch of numpy arrays into a BytesIO object and then load them back to a variable. So far, I can save the array's using following code

# Arrays
a = numpy.random.uniform(size=(10,10)) # dtype is float64
b = numpy.random.uniform(size=(10,10)) # dtype is float64

# Create bytes object
buf = io.BytesIO()

# Save the arrays into the buffer
numpy.savez_compressed(buf, a=a, b=b)

# Closed the buffer
buf.close()

I've been trying different methods to load them back. For example

ab = numpy.frombuffer(buf.read(), dtype='float64')

which raises ValueError: buffer size must be a multiple of element size. And trying to load as one would if it was a file

ab = numpy.load(buf)

raises ValueError: Cannot load file containing pickled data when allow_pickle=False and when buf.read() i get ValueError: embedded null byte.

Upvotes: 1

Views: 1588

Answers (1)

Vishal
Vishal

Reputation: 3296

The following works for me [Numpy version: numpy==1.16.1]:

import numpy
import io

# Arrays
a = numpy.random.uniform(size=(10,10)) # dtype is float64
b = numpy.random.uniform(size=(10,10)) # dtype is float64

# Create bytes object
buf = io.BytesIO()

# Save the arrays into the buffer
numpy.savez_compressed(buf, a=a, b=b)
buf.seek(0)

ab = numpy.load(buf)
print(ab['a'])
print(ab['b'])

# Closed the buffer
buf.close()

Upvotes: 1

Related Questions