Nanoseconds & Chrono C++

Question

2018-10-01 00:06:16.700000000

I have a time series data file with timestamps as above. I need to convert this to nanoseconds from epoch, then I will need to add milli, micro, or nanos to the timestamp (shift). Finally for select records bring it back to the format above.

I am having trouble with the time point creation - and how to represent nanoseconds ... It works fine with micros.

Can I please request some assistance with the code fragment below .... and also once I have the nanoseconds from epoch - how do I go back to a timestamp like the one above.

std::string ts("23 01 2020 20:59:59.123456789");

XXXX (ts);

void XXXXX (string timestamp)
{
     stringstream temp_ss(timestamp);

     tm temp_time_object = {};

     temp_ss >> get_time (&temp_time_object, "%Y/%m/%d %H:%M:%S");

     chrono::system_clock::time_point temp_time_point = system_clock::from_time_t(mktime(&temp_time_object));
    // chrono::high_resolution_clock::time_point temp_time_point1 = temp_time_point;

    auto  nsecs           =  stol (timestamp.substr (timestamp.find_first_of('.')+1,9));

// +++ This is where I GET stuck forming the time_point....
// I've tried this so many different ways .... 
// Is it that the system_clock cannot accept nanos? 


    temp_time_point += nanoseconds (nsecs);

     auto micro_from_epoch = duration_cast<nanoseconds> (temp_time_point.time_since_epoch()).count();

     cout << micro_from_epoch << endl;

}

Answer 1

I am having trouble with the time point creation - and how to represent nanoseconds ... It works fine with micros.

This tells me that your system_clock::time_point has a precision of coarser than nanoseconds (on llvm it is microseconds, on Windows, 1/10 microseconds). The easiest way to add nanoseconds to such a time_point is:

auto tp = temp_time_point + nanoseconds (nsecs);

This forms a time_point that is still system_clock based, but has the precision of the "common type" of system_clock::duration and nanoseconds, which in practice, will just be nanoseconds.

assume all timestamps are GMT, UTC + 0

Now the problem is that mktime converts from a local tm to a UTC time_t. But you are wanting to convert from a UTC field type, to a UTC serial type.

This is easily accomplished in C++20 (I know you don't have it yet, hear me out):

#include <chrono>
#include <iostream>
#include <sstream>

std::chrono::sys_time<std::chrono::nanoseconds>
XXXXX(std::string const& timestamp)
{
    using namespace std;
    using namespace std::chrono;
    istringstream temp_ss{timestamp};
    sys_time<nanoseconds> tp;
    temp_ss >> parse("%F %T", tp);
    return tp;
}

int
main()
{
    auto tp = XXXXX("2018-10-01 00:06:16.700000000");
    std::cout << tp.time_since_epoch().count() << "ns\n";
    std::string s = std::format("{:%F %T}", tp);
    std::cout << s << '\n';
}

This converts the string to a chrono::time_point<system_clock, nanoseconds>, which is evidently different from your system_clock::time_point only in that your system_clock::time_point has precision coarser than nanoseconds.

Then format is used to convert the time_point back into a string.

Output:

1538352376700000000ns
2018-10-01 00:06:16.700000000

Now I know that a fully conforming C++20 <chrono> is a rare thing these days (it's coming). Until it gets here, there is a C++20 <chrono> preview library that is compatible back to C++11. It is free and open source. And requires very few syntactic changes:

#include "date/date.h"
#include <chrono>
#include <iostream>
#include <sstream>

date::sys_time<std::chrono::nanoseconds>
XXXXX(std::string const& timestamp)
{
    using namespace date;
    using namespace std;
    using namespace std::chrono;
    istringstream temp_ss{timestamp};
    sys_time<nanoseconds> tp;
    temp_ss >> parse("%F %T", tp);
    return tp;
}

int
main()
{
    using namespace date;
    auto tp = XXXXX("2018-10-01 00:06:16.700000000");
    std::cout << tp.time_since_epoch().count() << "ns\n";
    std::string s = format("%F %T", tp);
    std::cout << s << '\n';
}

Output:

1538352376700000000ns
2018-10-01 00:06:16.700000000

Answer 2

You work convert directly from text to nanosecond resolution. There are essentially two key libraries:

• CCTZ by some Google engineers, though (like many projects) not a formally issued Google product

• date by Howard Hinnant who will probably answer here before I am done typing; his library is the basis of content for this in C++20

I have wrapped both for R (via Rcpp) and have plenty of examples. But there are also examples in the two repos so maybe start there?

So for lack of better immediate CCTZ examples here is one where the R package is used; you see the inputs:

R> library(RcppCCTZ)
R> example(parseDatetime)

prsDttR> ds <- getOption("digits.secs")

prsDttR> options(digits.secs=6) # max value

prsDttR> parseDatetime("2016-12-07 10:11:12",        "%Y-%m-%d %H:%M:%S")   # full seconds
[1] "2016-12-07 10:11:12 UTC"

prsDttR> parseDatetime("2016-12-07 10:11:12.123456", "%Y-%m-%d %H:%M:%E*S") # fractional seconds
[1] "2016-12-07 10:11:12.123456 UTC"

prsDttR> parseDatetime("2016-12-07T10:11:12.123456-00:00")  ## default RFC3339 format
[1] "2016-12-07 10:11:12.123456 UTC"

prsDttR> now <- trunc(Sys.time())

prsDttR> parseDatetime(formatDatetime(now + 0:4))               # vectorised
[1] "2020-05-01 02:16:27 UTC" "2020-05-01 02:16:28 UTC"
[3] "2020-05-01 02:16:29 UTC" "2020-05-01 02:16:30 UTC"
[5] "2020-05-01 02:16:31 UTC"

prsDttR> options(digits.secs=ds)
R> 

The parser function called is (and ignore the R related bits)

Rcpp::DatetimeVector parseDatetime(Rcpp::CharacterVector svec,
                                   std::string fmt = "%Y-%m-%dT%H:%M:%E*S%Ez",
                                   std::string tzstr = "UTC") {
    cctz::time_zone tz;
    load_time_zone(tzstr, &tz);
    sc::system_clock::time_point tp;
    cctz::time_point<cctz::sys_seconds> unix_epoch =
        sc::time_point_cast<cctz::sys_seconds>(sc::system_clock::from_time_t(0));

    // if we wanted a 'start' timer
    //sc::system_clock::time_point start = sc::high_resolution_clock::now();

    auto n = svec.size();
    Rcpp::DatetimeVector dv(n, tzstr.c_str());
    for (auto i=0; i<n; i++) {
        std::string txt(svec(i));

        if (!cctz::parse(fmt, txt, tz, &tp)) Rcpp::stop("Parse error on %s", txt);

        // time since epoch, with fractional seconds added back in
        // only microseconds guaranteed to be present
        double dt = sc::duration_cast<sc::microseconds>(tp - unix_epoch).count() * 1.0e-6;

        // Rcpp::Rcout << "tp: " << cctz::format(fmt, tp, tz) << "\n"
        //             << "unix epoch: " << cctz::format(fmt, unix_epoch, tz) << "\n"
        //             << "(tp - unix.epoch).count(): " << (tp - unix_epoch).count() << "\n"
        //             << "dt: " << dt << std::endl;

        dv(i) = Rcpp::Datetime(dt);
    }

    return dv;
}

It looks over the incoming vector svec of strings and converts each.

Edit: Here is another example using our nanotime package which leverages and uses the CCTZ parser:

R> library(nanotime)
R> as.nanotime("2020-01-29 13:12:00.000000001 America/New_York")
[1] 2020-01-29T18:12:00.000000001+00:00
R> 

Full 9 + 9 digits precision using underlying nanoseconds since epoch, fully interoperable with std::chrono.