CODEWITHSUNDEEP
apache-sparkapache-spark-sql
sri hari kali charan Tummala
sri hari kali charan Tummala

Reputation: 1134

spark sql current timestamp function

Is there a sql fucntion in spark sql which returns back current timestamp , example in impala NOW() is the function which returns back current timestamp is there similar in spark sql ?

Thanks

Upvotes: 21

Views: 50470

Answers (3)

Vikramraj
Vikramraj

Reputation: 198

You can use the following code to get the date and timestamp in Spark with Scala code.

import org.apache.spark.sql.functions._
val newDf = df.withColumn("current_date",current_date())
.withColumn("current_timestamp",current_timestamp())

The result will be something like this.

+------------+-----------------------+
|current_date|current_timestamp      |
+------------+-----------------------+
|2022-06-06  |2022-06-06 12:25:55.349|
+------------+-----------------------+

Upvotes: 0

Luis Estrada
Luis Estrada

Reputation: 449

It is possible to use Date and Timestamp functions from pyspark sql functions.

Example:

spark-sql> select current_timestamp();
2022-05-07 16:43:43.207
Time taken: 0.17 seconds, Fetched 1 row(s)
spark-sql> select current_date();
2022-05-07
Time taken: 5.224 seconds, Fetched 1 row(s)
spark-sql>

Reference: https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.functions.current_timestamp.html

Upvotes: 0

Kaushal
Kaushal

Reputation: 3367

Try current_timestamp function.

current_timestamp() - Returns the current timestamp at the start of query evaluation. All calls of current_timestamp within the same query return the same value.

Upvotes: 35

Related Questions