CODEWITHSUNDEEP
pythonnetcdfnetcdf4python-iris
user2986108
user2986108

Reputation: 105

Convert netCDF into CSV using python

I am new to Python and trying to parse the netCDF file into .CSV in columanar format, so I can load that data into RDBMS for other reporting purpose. Please refer the details below.

Sanpshot of my netCDF file:

dimensions:
time = UNLIMITED ; // (36 currently)
grid_latitude = 548 ;
grid_longitude = 421 ;
time_0 = UNLIMITED ; // (3 currently)
pressure = 3 ;
time_1 = UNLIMITED ; // (3 currently)
bnds = 2 ;
pressure_0 = 2 ;
pressure_1 = 3 ;
dim0 = UNLIMITED ; // (3 currently)
grid_longitude_0 = 421 ;
grid_latitude_0 = 547 ;
time_3 = UNLIMITED ; // (3 currently)
variables:
float stratiform_snowfall_rate(time, grid_latitude, grid_longitude) ;
stratiform_snowfall_rate:_FillValue = -1.073742e+09f ;
string stratiform_snowfall_rate:long_name = "stratiform_snowfall_rate" ;
string stratiform_snowfall_rate:units = "kg m-2 s-1" ;
string stratiform_snowfall_rate:um_stash_source = "m01s04i204" ;
string stratiform_snowfall_rate:grid_mapping = "rotated_latitude_longitude" ;string stratiform_snowfall_rate:coordinates = "forecast_period forecast_reference_time" ;int rotated_latitude_longitude ;

My Code:

from netCDF4 import Dataset, num2date
filename ='prods_op_mogreps-uk_20140717_03_11_015.nc'
nc = Dataset(filename, 'r', Format='NETCDF4')
 ncv = nc.variables
 lats = nc.variables['grid_latitude'][:]
 lons = nc.variables['grid_longitude'][:]
 sfc= nc.variables['stratiform_snowfall_rate'][:]
 times = nc.variables['time'][:]
 units = nc.variables['time'].units
 dates = num2date (times[:], units=units, calendar='365_day')
 header = ['Latitude', 'Longitude']
 for d in dates:
    header.append(d)
import csv
with open('output.csv', 'wb') as csvFile:
    outputwriter = csv.writer(csvFile, delimiter=',')
    for time_index, time in enumerate(times): # pull the dates out for the header
         t = num2date(time, units = units, calendar='365_day')
         header.append(t)
    outputwriter.writerow(header)  
    for lat_index, lat in enumerate(lats):
        content = lat
        #print lat_index
        for lon_index, lon in enumerate(lons):
            content.append(lon)
            #print lon_index    
            for time_index, time in enumerate(times): # for a date
                # pull out the data 
                data = sfc[time_index,lat_index,lon_index]
                content.append(data)
                outputwriter.writerow(content)
csvFile.close()
nc.close()

I am getting below error:

TypeError                                 Traceback (most recent call last)
<ipython-input-41-b4b3b888999f> in <module>
      4          t = num2date(time, units = units, calendar='365_day')
      5          header.append(t)
----> 6     outputwriter.writerow(header)
      7     for lat_index, lat in enumerate(lats):
      8         content = lat

TypeError: a bytes-like object is required, not 'str'

Please help me on this code. Thanks

Upvotes: 0

Views: 2312

Answers (2)

Robert Wilson
Robert Wilson

Reputation: 3417

The easiest method for this would be to use xarray and pandas.

import xarray as xr
import pandas as pd

You will first need to read in the data using xarray:

data = xr.open_dataset(filename)

This will then need to be converted to a pandas dataset, with the index reset:

data_df = data.to_dataframe().reset_index()

Finally, you would need to save this as a csv:

data_df.to_csv(outfile)

Upvotes: 1

SpghttCd
SpghttCd

Reputation: 10890

You open your output file in binary mode by choosing 'wb'.
Therefore the file writing function expects binary data, i.e. a bytes object.

However, as you're requesting help for writing a csv-file, I assume you want to write plain text data, so you simply should remove the b for binary here:

with open('output.csv', 'w') as csvFile:

Upvotes: 1

Related Questions