CODEWITHSUNDEEP
rdata.table
David
David

Reputation: 10232

R Milliseconds in Strptime data.table

I am trying to convert a date and a time to a timestamp with milliseconds in R with data.table (I have ~300M rows, therefore speed does matter! So if you have a faster option than strptime (for example a vectorised function where I don't need a 'by' argument in the data.table), I would be delighted! :) ).

The issue so far is the following:

library(data.table)
options(digits.secs = 3)
a <- data.table(day = "20150727", time = "11:10:05.016")

a[, mtime := strptime(paste(day, time), "%Y%m%d %H:%M:%OS", tz = "GMT"),
  by = 1:nrow(a)]
a
#> a
#   day       time        mtime
#1: 20150727 11:10:05.016 5.016

The issue is that mtime is definitely not right... I want to get the whole timestamp and not just the seconds. When I do it by hand it works fine:

strptime(paste("20150727", "11:10:05.016"), "%Y%m%d %H:%M:%OS", tz = "GMT")
# [1] "2015-07-27 11:10:05.016 GMT"

Any ideas? Thank you!

Upvotes: 1

Views: 703

Answers (1)

Nikos
Nikos

Reputation: 3297

I experienced the same issue you are describing. However, when I use as.POSIXct, there is no issue. Can you try yourself and see if it works?

I tried the following:

a[, posixct:=as.POSIXct(paste(day, time),format="%Y%m%d %H:%M:%OS", tz = "GMT")]

    day         time mtime                 posixct
 1: 20150727 11:10:05.016 5.016 2015-07-27 11:10:05.016

Update

Following the comments of both Davids, I spent some time investigating the fasttime package. Indeed it is much faster than as.POSIXct, however it comes with a minor requirement, that is that the date format has to be the standard YYYY-MM-DD. Below is the code to recreate the table and some time comparisons.

# assuming that day is 2015-07-27
a[, fastposixct:=fastPOSIXct(paste(day, time),required.components = 6L, tz = "GMT")]

Unit: microseconds
expr          min    lq     mean median      uq     max neval
as.POSIXct 61.579 62.64 69.59851 63.349 65.4725 240.298   100

Unit: microseconds
expr           min     lq     mean median      uq     max neval
fastPOSIXct 26.897 27.959 33.96092 28.666 30.6135 135.544   100

Hope this helps.

Upvotes: 4

Related Questions