CODEWITHSUNDEEP
rapache-sparksparkr
Ole Petersen
Ole Petersen

Reputation: 680

Can't use 'date_add'-function in sparkR

In SparkR (spark-1.4.0) I want to use the date_add function. https://spark.apache.org/docs/latest/api/R/

But when I type it in SparkR it says that the function does not exist. How can that be and how can I get it ?

Upvotes: 0

Views: 320

Answers (2)

zero323
zero323

Reputation: 330343

In 1.4.0 it can be invoked using raw SQL:

library(magrittr)

hiveContext <- sparkRHive.init(sc)

df <- createDataFrame(hiveContext, data.frame(ts=c("2015-12-01"))) %>%  
    select(alias(cast(.$ts, "date"), "ts"))

df %>% selectExpr("ts", "date_add(ts, 1) AS next_day") %>% head

##          ts   next_day
## 1 2015-12-01 2015-12-02

Upvotes: 1

Roman Luštrik
Roman Luštrik

Reputation: 70653

The documentation you link to is for version 1.5.2. It is possible that your version may not have this function yet. According to comments below you will need to update Spark and SparkR.

Upvotes: 2

Related Questions