CODEWITHSUNDEEP
scaladataframeapache-spark
louis choi
louis choi

Reputation: 23

spark - where dataframe is defined in scala source code

Im trying to find DataFrame class definition in scala source code not in pyspark.

There is some files like DataFrameReader, DataFrameWriter, Dataset But not DataFrame.

I have found some directories such as spark/sql, spark/core.

Upvotes: 2

Views: 379

Answers (1)

manojlds
manojlds

Reputation: 301047

A DataFrame is just a Dataset[Row] and is a type alias:

type DataFrame = Dataset[Row]

https://github.com/apache/spark/blob/50538600ec972469338370f7e2d3674ca8b3c389/sql/core/src/main/scala/org/apache/spark/sql/package.scala#L46

Upvotes: 4

Related Questions