CODEWITHSUNDEEP
pythonarraysnumpy
astrogirl79
astrogirl79

Reputation: 69

How to load multiple columns with numpy loadtext?

I am trying to load about 10 columns of data into numpy loadtxt. I want each column to be a separate numpy array. How can I do this? Only two columns are actually being imported, I need all 10. Here's my code:

import sys
import numpy as np 
import scipy.stats

data = np.loadtxt('AD_hw6.txt')

p = data[:,0] #pressure in hpa
z = data[:,1] #height in m
t = data[:,2] #tempertature in degrees celcius
dp = data[:,3] #dewpoint in degrees celcius
rh = data[:,4] #relative humidity (%)
mr = data[:,5] #mixing ratio in g/kg

This is only 5 but I'm using 10.

Upvotes: 3

Views: 7770

Answers (3)

GoldenGenova
GoldenGenova

Reputation: 54

I think the problem is that the first column row only has 2 values; I suggest you fill in zeros or NaN for the empty rows in the columns.

Upvotes: 0

hpaulj
hpaulj

Reputation: 231325

With a copy-n-paste from the WY link I create a file with

-----------------------------------------------------------------------------
   PRES   HGHT   TEMP   DWPT   RELH   MIXR   DRCT   SKNT   THTA   THTE   THTV
    hPa     m      C      C      %    g/kg    deg   knot     K      K      K
-----------------------------------------------------------------------------
 1000.0    116
  971.0    357    8.0    6.0     87   6.07    200      5  283.5  300.6  284.6
  956.0    487    7.2    5.0     86   5.75    215     18  284.0  300.2  285.0
  942.1    610    8.0    5.5     84   6.06    230     30  286.0  303.2  287.1
  933.0    691    8.6    5.9     83   6.28    233     30  287.4  305.3  288.5
  925.0    763    8.0    4.9     81   5.90    235     30  287.5  304.4  288.5
  908.2    914    7.2    3.2     76   5.32    245     29  288.2  303.5  289.1
 ...

I can load it as a 2d array with:

In [1]: data = np.genfromtxt('stack46636938.txt',skip_header=5)
In [2]: data.shape
Out[2]: (35, 11)
In [3]: data[:3]
Out[3]: 
array([[ 971.  ,  357.  ,    8.  ,    6.  ,   87.  ,    6.07,  200.  ,
           5.  ,  283.5 ,  300.6 ,  284.6 ],
       [ 956.  ,  487.  ,    7.2 ,    5.  ,   86.  ,    5.75,  215.  ,
          18.  ,  284.  ,  300.2 ,  285.  ],
       [ 942.1 ,  610.  ,    8.  ,    5.5 ,   84.  ,    6.06,  230.  ,
          30.  ,  286.  ,  303.2 ,  287.1 ]])

Notice that I skipped the line with only two values.

Now I can do

p = data[:,0]

I could also use the unpack option

In [7]: p,z,t = np.genfromtxt('stack46636938.txt',skip_header=5,unpack=True,usecols=range(3))

This is the equivalent of using Python unpacking with the transpose of the 2d array:

In [9]: p,z,t = data[:,:3].T

I could also load the data as a structured array with:

In [11]: names = '   PRES   HGHT   TEMP   DWPT   RELH   MIXR   DRCT   SKNT   THT
    ...: A   THTE   THTV'.split()
In [12]: names
Out[12]: 
['PRES',
 'HGHT',
 'TEMP',
 'DWPT',
 ....
 'THTV']
In [13]: data = np.genfromtxt('stack46636938.txt', skip_header=5, dtype=None, names=names)
In [14]: data[:3]
Out[14]: 
array([( 971. , 357,  8. ,  6. , 87,  6.07, 200,  5,  283.5,  300.6,  284.6),
       ( 956. , 487,  7.2,  5. , 86,  5.75, 215, 18,  284. ,  300.2,  285. ),
       ( 942.1, 610,  8. ,  5.5, 84,  6.06, 230, 30,  286. ,  303.2,  287.1)],
      dtype=[('PRES', '<f8'), ('HGHT', '<i4'), ('TEMP', '<f8'), ('DWPT', '<f8'), ('RELH', '<i4'), ('MIXR', '<f8'), ('DRCT', '<i4'), ('SKNT', '<i4'), ('THTA', '<f8'), ('THTE', '<f8'), ('THTV', '<f8')])

and access fields by name

In [15]: data['PRES']
Out[15]: 
array([ 971. ,  956. ,  942.1,  933. ,  925. ,  908.2,  900. ,  875. ,
        850. ,  842.8,  827. ,  811.4,  792. ,  786. ,  781. ,  776. ,
        752.1,  736. ,  726. ,  724.1,  721. ,  714. ,  710. ,  706. ,
        703. ,  701. ,  700. ,  690. ,  671. ,  670.8,  669. ,  666. ,
        662. ,  645.4,  639. ])

loadtxt also works with the right row skip and unpack:

np.loadtxt('stack46636938.txt',skiprows=5,unpack=True)

If the short row is included I get an error. It starts expecting two columns and then objects when it gets 11.

Upvotes: 2

GoldenGenova
GoldenGenova

Reputation: 54

I think the problem is that the first column row only has 2 values; I suggest you fill in zeros or NaN for the empty rows in the columns.

Upvotes: 0

Related Questions