CODEWITHSUNDEEP
scalaapache-spark
vishan tahlani
vishan tahlani

Reputation: 47

How do i write a text(.txt) file to a dataframe without using RDDs and print it on a console

I wanted to write a .txt file to a dataframe and print it on console

I have tried it printing in the same manner as i was printing a .csv file to the console through dataframes

def txtreader():DataFrame={

val loadTxt=spark.read
            .format("txt")
  .option("header","true")
  .load("C:\\Users\\1591532\\Spark-Learning-Workspace\\Text Files\\abc.txt")

return loadTxt

I am getting an error "Failed to find data source: txt."

Upvotes: 0

Views: 452

Answers (2)

Ana
Ana

Reputation: 831

For Spark 1.6 and higher, you can use the csv data source:

val df = spark.read.csv("file.txt")

For your case, You can also specify header option, delimeter etc e.g.:

val df = spark.read.option("header", "true").option("delimiter", ";").csv("file.txt")

Upvotes: 1

Ravi
Ravi

Reputation: 470

Please find the below code to read the text file.

scala> spark.read.text("sample.txt")
res34: org.apache.spark.sql.DataFrame = [value: string]

scala> res34.show
+-----+
|value|
+-----+
|  abc|
+-----+

Upvotes: 0

Related Questions