CODEWITHSUNDEEP
scalaapache-spark
Gaurang Shah
Gaurang Shah

Reputation: 12900

scala - how to avoid using mutable list and casting

I have written the following code which works fine. However, I wanted to check if there is a way I can avoid using mutable list and casting datatype from any to string.

import scala.collection.mutable.ListBuffer

val databases = spark.catalog.listDatabases.select($"name").collect().map(_(0)).toList
var tables= new ListBuffer[String]()
databases.foreach{database =>
  val t = spark.catalog.listTables(database.asInstanceOf[String]).filter($"isTemporary" === false).filter($"tableType" =!= "VIEW").select($"name").collect.map(database+"."+_(0).asInstanceOf[String]).toList
  tables = tables ++ t
}
tables.foreach(println)

Upvotes: 0

Views: 54

Answers (1)

Dima
Dima

Reputation: 40500

You can use row deconstructor to get rid of casting:

   val databases: List[String] = listDatabases.select($"name").collect().map { 
      case Row(name: String) => name
   }.to[List]

As for mutability, just use flatMap instead of foreach:

val tables = databases.flatMap { db => listTables(db).filter .... }

Upvotes: 2

Related Questions