CODEWITHSUNDEEP
scalaapache-sparkapache-spark-sql
yjxyjx
yjxyjx

Reputation: 121

Fastest way to check if DataFrame(Scala) is empty?

How to check if DataFrame(Scala) is empty in fastest way?I use DF.limit(1).rdd.isEmpty, faster than DF.rdd.isEmpty,but not ideal.Is there any better way to do that?

Upvotes: 3

Views: 9215

Answers (1)

Ton Torres
Ton Torres

Reputation: 1529

I usually wrap a call to first around a Try:

import scala.util.Try

val t = Try(df.first)

From there you can match on it if it's a Success or Failure to control logic:

import scala.util.{Success,Failure}

t match {
  case Success(df) => //do stuff with the dataframe

  case Failure(e) => 
    // dataframe is empty; do other stuff
    //e.getMessage will return the exception message
}

Upvotes: 3

Related Questions