THIS USER NEEDS HELP
THIS USER NEEDS HELP

Reputation: 3266

Is `dataframe.write.mode(SaveMode.Overwrite)text(output)` valid Scala/Spark SQL syntax?

So I had a typo of dataframe.write.mode(SaveMode.Overwrite)text(output) missing a period between mode and text, but eclipse doesn't seem to complain, and when I run it through Junit and production, everything seems to run fine without exception, even producing correct output. I am confused that there wasn't any bug, and my Spark DAG does show that my code has changed, so I am more confused. Any ideas?

Upvotes: 1

Views: 2246

Answers (1)

cheseaux
cheseaux

Reputation: 5315

It is just the way scala works. It's the Infix notation

A white space is not required because of the parentheses. Here is a demonstration :

scala> val l = List(1,2,3)
l: List[Int] = List(1, 2, 3)

scala> l.take(1)
res4: List[Int] = List(1)

scala> l take 1
res5: List[Int] = List(1)

scala> (l)take 1
res6: List[Int] = List(1)

scala> l.take(2)take(1)
res7: List[Int] = List(1)

Upvotes: 4

Related Questions