CODEWITHSUNDEEP
pythonarraysnumpyreshapelarge-data
Sagnik
Sagnik

Reputation: 71

How do I stack multiple columns from one single array on top of each other in Python?

For example, the shape of the array is 5,4.

a = np.random.randint(10, size= (5, 4))

a = 
[[1 4 5 0]
 [3 1 5 1]
 [4 8 0 9]
 [8 1 5 8]
 [6 4 7 4]]

I want the array to be reshaped as:

a = 
[[1 4]
 [3 1]
 [4 8]
 [8 1]
 [6 4]
 [5 0]
 [5 1]
 [0 9]
 [5 8]
 [7 4]]

My original array size is around 200 GB and of shape 80000*480600. I have tried to use remap mode but it is very slow.

Upvotes: 2

Views: 352

Answers (3)

hpaulj
hpaulj

Reputation: 231385

This can be done with a reshape and transpose (and a final reshape)

In [195]: arr = np.arange(20).reshape(5,4)                                                                   
In [196]: arr                                                                                                
Out[196]: 
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11],
       [12, 13, 14, 15],
       [16, 17, 18, 19]])
In [197]: arr.reshape(5,2,2)                                                                                 
Out[197]: 
array([[[ 0,  1],
        [ 2,  3]],

       [[ 4,  5],
        [ 6,  7]],

       [[ 8,  9],
        [10, 11]],

       [[12, 13],
        [14, 15]],

       [[16, 17],
        [18, 19]]])
In [198]: arr.reshape(5,2,2).transpose(1,0,2)                                                                
Out[198]: 
array([[[ 0,  1],
        [ 4,  5],
        [ 8,  9],
        [12, 13],
        [16, 17]],

       [[ 2,  3],
        [ 6,  7],
        [10, 11],
        [14, 15],
        [18, 19]]])

Identifying the right transpose (or swapaxes) may require a bit of trial and error.

In [199]: arr.reshape(5,2,2).transpose(1,0,2).reshape(-1,2)                                                  
Out[199]: 
array([[ 0,  1],
       [ 4,  5],
       [ 8,  9],
       [12, 13],
       [16, 17],
       [ 2,  3],
       [ 6,  7],
       [10, 11],
       [14, 15],
       [18, 19]])

The equivalent with split and concatenate:

In [200]: np.concatenate(np.hsplit(arr,2), axis=0)                                                           
Out[200]: 
array([[ 0,  1],
       [ 4,  5],
       [ 8,  9],
       [12, 13],
       [16, 17],
       [ 2,  3],
       [ 6,  7],
       [10, 11],
       [14, 15],
       [18, 19]])

The transpose route should be faster.

Upvotes: 0

Andy L.
Andy L.

Reputation: 25239

hsplit and reshape also works

np.reshape(np.hsplit(a, 2), (-1,2))

Out[99]:
array([[1, 4],
       [3, 1],
       [4, 8],
       [8, 1],
       [6, 4],
       [5, 0],
       [5, 1],
       [0, 9],
       [5, 8],
       [7, 4]])

Upvotes: 0

Vicrobot
Vicrobot

Reputation: 3988

Use numpy.hsplit and numpy.concatenate:-

>>> a = np.random.randint(10, size= (5, 4))
>>> a
array([[8, 5, 8, 9],
       [9, 5, 6, 3],
       [5, 3, 8, 7],
       [9, 0, 9, 9],
       [0, 7, 8, 0]])
>>> t = np.hsplit(a, 2)
>>> t
[array([[8, 5],
       [9, 5],
       [5, 3],
       [9, 0],
       [0, 7]]), array([[8, 9],
       [6, 3],
       [8, 7],
       [9, 9],
       [8, 0]])]
>>> np.concatenate([t[0], t[1]])
array([[8, 5],
       [9, 5],
       [5, 3],
       [9, 0],
       [0, 7],
       [8, 9],
       [6, 3],
       [8, 7],
       [9, 9],
       [8, 0]])

Upvotes: 2

Related Questions