CODEWITHSUNDEEP
pythonpandasnetcdfpython-xarray
Bruno
Bruno

Reputation: 1459

xarray and netCDF file with empty variables and 0-dimensional object dataframe

I'm trying to convert some .nc files into pandas dataframes using xarray.

Here's one of the netCDF files:

ftp://l5ftl01.larc.nasa.gov/MISR/MIL2ASAE.003/2017.08.31/MISR_AM1_AS_AEROSOL_P006_O094165_F13_0023.nc

And the code:

import xarray as xr
ds = xr.open_dataset("MISR_AM1_AS_AEROSOL_P006_O094165_F13_0023.nc")
df = ds.to_dataframe()

And the error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\abreucbr\AppData\Local\Continuum\anaconda3\envs\py36\lib\site-packages\xarray\core\
dataset.py", line 3088, in to_dataframe
    return self._to_dataframe(self.dims)
  File "C:\Users\abreucbr\AppData\Local\Continuum\anaconda3\envs\py36\lib\site-packages\xarray\core\
dataset.py", line 3078, in _to_dataframe
    index = self.coords.to_index(ordered_dims)
  File "C:\Users\abreucbr\AppData\Local\Continuum\anaconda3\envs\py36\lib\site-packages\xarray\core\
coordinates.py", line 80, in to_index
    raise ValueError('no valid index for a 0-dimensional object')
ValueError: no valid index for a 0-dimensional object

If I inspect the ds variable, for example,

ds.variables

I get

Frozen(OrderedDict())

The .nc file has a few MB, so it doesn't seem to be "empty".

What's the problem here?

Upvotes: 3

Views: 2545

Answers (2)

kmuehlbauer
kmuehlbauer

Reputation: 461

You can inspect the nc-file using netcdf4 Dataset. This gives you insight in the complete structure including groups.

Upvotes: 1

jhamman
jhamman

Reputation: 6434

Your dataset seems to be setup with a hierarchy of groups. Xarray's open_dataset function only supports opening a single group at a time. So you'll need to open just one group at a time. Something like:

xr.open_dataset("MISR_AM1_AS_AEROSOL_P006_O094165_F13_0023.nc", group='4.4_KM_PRODUCTS')

Generally speaking, the to_dataframe method is going to have limited utility for your dataset since collapsing 6 dimensions into a single index is going to be pretty clunky/inefficient.

Upvotes: 4

Related Questions