CODEWITHSUNDEEP
pythonnumpy
Awais Hussain
Awais Hussain

Reputation: 442

How to read multiple numpy arrays from a folder

I have multiple numpy arrays (.npy) in a folder. Is there a way to read all of them automatically in Python? Or do I need to enter their names manually? I couldn't find information related to reading multiple numpy arrays from a folder for Python.

Upvotes: 3

Views: 5134

Answers (2)

trsvchn
trsvchn

Reputation: 8981

You can use glob to grab all the .npy files matching a specified *.npy pattern. glob.glob returns a list of pathnames and glob.iglob returns an iterator instead of storing all the pathnames simultaneously (will be useful if you have a huge amount of files). Here is a small example:

Code:

import os
import glob
import numpy as np


# Let's create folder
folder = './np_arrays'
try: 
    os.mkdir(folder)
except OSError: 
    print('Folder exists!')

# Some dummy arrays
a = np.zeros((1, 5))
b = np.ones((1, 5))

# Save them as .npy to the created folder
np.save(os.path.join(folder, 'a'), a)
np.save(os.path.join(folder, 'b'), b)

# Getting all the numpy arrays .npy files based on matching pattern (*.npy)
file_paths = glob.glob(os.path.join(folder, '*.npy'))
print(file_paths)

# Import arrays from folder and store them as a dict
array_dict = {os.path.basename(f)[0]: np.load(f) for f in file_paths}
print(array_dict)

Output:

['./np_arrays/a.npy', './np_arrays/b.npy']
{'a': array([[0., 0., 0., 0., 0.]]), 'b': array([[1., 1., 1., 1., 1.]])}

Upvotes: 4

Iain Shelvington
Iain Shelvington

Reputation: 32244

You can always loop over all files that have the extension you are looking for and store them in a data structure that makes sense for your use case

For example to store them all in a dictionary where the keys are the filenames:

arrays = {}
for filename in os.listdir(dir):
    if filename.endswith('.npy'):
        arrays[filename] = load_array(filename)

Upvotes: 3

Related Questions