CODEWITHSUNDEEP
pythonarraysnumpy
CatarinaCM
CatarinaCM

Reputation: 1245

Load and concatenate numpy arrays

I have several numpy arrays, and I want to read them and concatenate them all together:

For loading the files I did the following:

import numpy as np
AVERAGE_files = glob.glob('*AVERAGE*')
for AV in AVERAGE_files:
    x = np.load(AV)

Now for the concatenation I know that I have to use the np.concatenate function, but I don't know how to start the concatenation. Do I have to declare a np.zeros array outside the cycle?

Upvotes: 3

Views: 1811

Answers (4)

kory
kory

Reputation: 512

This is quite fast and very pythonic compared to the other answers

import numpy as np
AVERAGE_files = glob.glob('*AVERAGE*')
x_averages = np.concatenate([np.load(average_file) for average_file in AVERAGE_files], dtype=float)

Upvotes: 0

AGML
AGML

Reputation: 920

Numpy arrays cannot be appended to, only copied. Thus np.concatenate results in a new array which is identical to the old except the extra values. Constructing an array by successive concatenation, while certainly possible, is not an efficient approach.

If the size of the files is known you are almost certainly better off allocating an array with np.zeros out of cycle, as you suggest:

import numpy as np
array = np.zeros([size])
for i in range(0,len(AVERAGE_files)):
    with open(AVERAGE_files[i]) as f: #assuming AV are strings?
       array[i] = map(float,f)

Alternatively you can build a list (which can be appended to efficiently) and then convert it to an array at the end:

import numpy as np
array = np.asarray([map(float,open(AV)) for AV in AVERAGE_files])

Upvotes: 1

areuexperienced
areuexperienced

Reputation: 2071

Surely just something like this?

import numpy as np

new = np.array([])

for AV in AVERAGE_files:
   x = np.load(AV) # I assume num in your example is an alias for numpy?
   new = np.concatenate([new,x])

Upvotes: 4

Andrew Jaffe
Andrew Jaffe

Reputation: 27077

If you already have the individual arrays loaded as x1, x2, x3, you can use

x = numpy.vstack((x1, x2, x3))

but it's probably more efficient to use something like @areuexperienced's concatenation technique since you don't keep the individual x1s.

Upvotes: 2

Related Questions