How to calculate daily average from ERA5 hourly netCDF data?

Question

Hi dear all,

I do apologize for repeating the question. I have downloaded and merged the ERA5 hourly dew-point temperature data (d2m_wb.nc) from the Copernicus web platform. Now, I want to calculate the daily mean from the hourly d2m_wb.nc data. The timestamps are 00, 01, 02...23. The ECMWF provided an example for the calculation of daily total precipitation (https://confluence.ecmwf.int/display/CKB/ERA5%3A+How+to+calculate+daily+total+precipitation). It said to cover total precipitation for 1st January 2017, we need two days of data. (a) 1st January 2017 time = 01 - 23 will give you total precipitation data to cover 00 - 23 UTC for 1st January 2017 (b) 2nd January 2017 time = 00 will give you total precipitation data to cover 23 - 24 UTC for 1st January 2017

That means I need to shift the -1hour timestamp to account for step (b). Accordingly, I did it using Climate Data Operators (CDO).

cdo daymean -shifttime,-1hour in.nc out.nc

and got the following result.

cdo sinfo d2m_wb.nc
   File format : NetCDF2
    -1 : Institut Source   T Steptype Levels Num    Points Num Dtype : Parameter ID
     1 : unknown  unknown  v instant       1   1       475   1  F64  : -1
   Grid coordinates :
     1 : lonlat                   : points=475 (19x25)
                              lon : 85.5 to 90 by 0.25 degrees_east
                              lat : 21.5 to 27.5 by 0.25 degrees_north
   Vertical coordinates :
     1 : surface                  : levels=1
   Time coordinate :  25904 steps
     RefTime =  1900-01-01 00:00:00  Units = hours  Calendar = gregorian  Bounds = true
  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss  YYYY-MM-DD hh:mm:ss
  1949-12-31 23:00:00  1950-01-01 11:00:00  1950-01-02 11:00:00  1950-01-03 11:00:00
  1950-01-04 11:00:00  1950-01-05 11:00:00  1950-01-06 11:00:00  1950-01-07 11:00:00
  1950-01-08 11:00:00  1950-01-09 11:00:00  1950-01-10 11:00:00  1950-01-11 11:00:00
  1950-01-12 11:00:00  1950-01-13 11:00:00  1950-01-14 11:00:00  1950-01-15 11:00:00
  1950-01-16 11:00:00  1950-01-17 11:00:00  1950-01-18 11:00:00  1950-01-19 11:00:00
  1950-01-20 11:00:00  1950-01-21 11:00:00  1950-01-22 11:00:00  1950-01-23 11:00:00
  1950-01-24 11:00:00  1950-01-25 11:00:00  1950-01-26 11:00:00  1950-01-27 11:00:00
  1950-01-28 11:00:00  1950-01-29 11:00:00  1950-01-30 11:00:00  1950-01-31 11:00:00
  1950-02-01 11:00:00  1950-02-02 11:00:00  1950-02-03 11:00:00  1950-02-04 11:00:00
  1950-02-05 11:00:00  1950-02-06 11:00:00  1950-02-07 11:00:00  1950-02-08 11:00:00
  1950-02-09 11:00:00  1950-02-10 11:00:00  1950-02-11 11:00:00  1950-02-12 11:00:00
  1950-02-13 11:00:00  1950-02-14 11:00:00  1950-02-15 11:00:00  1950-02-16 11:00:00
  1950-02-17 11:00:00  1950-02-18 11:00:00  1950-02-19 11:00:00  1950-02-20 11:00:00
  1950-02-21 11:00:00  1950-02-22 11:00:00  1950-02-23 11:00:00  1950-02-24 11:00:00
  1950-02-25 11:00:00  1950-02-26 11:00:00  1950-02-27 11:00:00  1950-02-28 11:00:00
   ................................................................................
   ................................................................................
   ................................................................................
   .................
  2020-10-03 11:00:00  2020-10-04 11:00:00  2020-10-05 11:00:00  2020-10-06 11:00:00
  2020-10-07 11:00:00  2020-10-08 11:00:00  2020-10-09 11:00:00  2020-10-10 11:00:00
  2020-10-11 11:00:00  2020-10-12 11:00:00  2020-10-13 11:00:00  2020-10-14 11:00:00
  2020-10-15 11:00:00  2020-10-16 11:00:00  2020-10-17 11:00:00  2020-10-18 11:00:00
  2020-10-19 11:00:00  2020-10-20 11:00:00  2020-10-21 11:00:00  2020-10-22 11:00:00
  2020-10-23 11:00:00  2020-10-24 11:00:00  2020-10-25 11:00:00  2020-10-26 11:00:00
  2020-10-27 11:00:00  2020-10-28 11:00:00  2020-10-29 11:00:00  2020-10-30 11:00:00
  2020-10-31 11:00:00  2020-11-01 11:00:00  2020-11-02 11:00:00  2020-11-03 11:00:00
  2020-11-04 11:00:00  2020-11-05 11:00:00  2020-11-06 11:00:00  2020-11-07 11:00:00
  2020-11-08 11:00:00  2020-11-09 11:00:00  2020-11-10 11:00:00  2020-11-11 11:00:00
  2020-11-12 11:00:00  2020-11-13 11:00:00  2020-11-14 11:00:00  2020-11-15 11:00:00
  2020-11-16 11:00:00  2020-11-17 11:00:00  2020-11-18 11:00:00  2020-11-19 11:00:00
  2020-11-20 11:00:00  2020-11-21 11:00:00  2020-11-22 11:00:00  2020-11-23 11:00:00
  2020-11-24 11:00:00  2020-11-25 11:00:00  2020-11-26 11:00:00  2020-11-27 11:00:00
  2020-11-28 11:00:00  2020-11-29 11:00:00  2020-11-30 11:00:00  2020-12-31 23:00:00
cdo    sinfo: Processed 1 variable over 25904 timesteps [6.03s 37MB

In this case, the timestep shows 11:00:00 (from 1950-01-01 onwards). I guess it should be 12:00:00. What wrong I've done here? Any suggestions will highly be appreciated? Thank you.

Answer 1

I also have similar issue with GLDAS 3-hourly temperature data.

Let say I use data for 1948, and the first data will be GLDAS_NOAH025_3H.A19480101.0300.020.nc4 which is temperature value for 1948-01-01 00:00:00 -- 1948-01-01 03:00:00 and the last data with year 1948 written on the filename is GLDAS_NOAH025_3H.A19481231.2100.020.nc4 which is temperature value for 1948-12-31 18:00:00 -- 1948-12-31 21:00:00

I added GLDAS_NOAH025_3H.A19490101.0000.020.nc4 into 1948 folder and merge all files into single netcdf using:

cdo mergetime *.nc4 merge_1948.nc4

Then I try to calculate the daily mean using:

cdo daymean merge_1948.nc4 tmean_1948.nc4

Unfortunately the total file (time) is 367, and the first data is 1948-01-01 00:00:00 -- 1948-01-01 21:00:00 and the last data is 1948-12-31 21:00:00 -- 1949-01-01 00:00:00

So, I tried to use shifttime and it solved the problem.

cdo daymean -shifttime,-3hour merge_1948.nc4 temp.nc4
cdo -shifttime,3hour temp.nc4 tmean_1948.nc4

and the first data is 1948-01-01 00:00:00 -- 1948-01-02 00:00:00 and the last data is 1948-12-31 00:00:00 -- 1949-01-01 00:00:00

Answer 2

Robert Wilson's answer is correct, I just wanted to quickly clarify that the confusion here is due to the difference between

• Instantaneous fields: such as clouds, water vapour, temperatures, winds, etc, these are fields that are valid for the instant
• Accumulated fields: such as radiative fluxes, latent and sensible heat fluxes, precipitation and so on, these are accumulated over a period of time, and the time stamp is placed at the end of the window.

Thus for instant fields Robert is correct that you don't want to shift, if you consider 00Z to be in the subsequent day, but you could equally validly argue that midnight should be included in the previous day (thus you would need shift), as it lies on the border. Convention says you don't shift, and count 00...23 as one day...

Concerning the fluxes, there are also more details in this post: Calculating ERA5 Daily Total Precipitation using CDO

Answer 3

This output appears correct. CDO has to make a decision about which timestep to use when averaging. In this case it takes the mid-point of each day, which is 11:00.

You'll notice that in the first day the time is 23:00, as there is only one time.

However, it is not clear why you would want to shift the time back one hour. Your code is not actually calculating the daily mean. Instead it is the mean of the final 23 hours of one day and the first hour of the next. Just change your CDO call to the following and everything should be fine:

cdo daymean in.nc out.nc