CODEWITHSUNDEEP
scalaapache-sparkgroup-byapache-spark-sql
AKC
AKC

Reputation: 1023

Spark DataFrame groupBy

I have Spark Java that looked like this. Code pulls data from oracle table using JDBC and displays the groupby output.

DataFrame jdbcDF = sqlContext.read().format("jdbc").options(options).load();
jdbcDF.show();   
jdbcDF.groupBy("VA_HOSTNAME").count().show();

Long ll = jdbcDF.count();
System.out.println("ll="+ll);

When I ran the code, jdbcDF.show(); is working, whereas the groupBy and count are not printing anything and no errors were thrown.

My column name is correct. I tried by printing that column and it worked, but when groupBy it's not working.

Can someone help me with DataFrame output? I am using spark 1.6.3.

Upvotes: 3

Views: 15630

Answers (1)

mrsrinivas
mrsrinivas

Reputation: 35434

You can try

import org.apache.spark.sql.functions.count

jdbcDF.groupBy("VA_HOSTNAME").agg(count("*")).show()

Upvotes: 4

Related Questions