CODEWITHSUNDEEP
google-cloud-dataproc
Thomas Oldervoll
Thomas Oldervoll

Reputation: 340

Spark 2.0 and Scala 2.11 on Dataproc?

Spark 2.0 final was released last week - when can we expect to see it on Dataproc? Spark 2.0 uses Scala 2.11 as default - does this mean that Dataproc will use Scala 2.11?

Scala 2.11 is actually the feature we are most eagerly awaiting. Scala 2.10 is getting old and we have a couple of libraries we would like to use that we cannot because they don't support 2.10.

Upvotes: 1

Views: 405

Answers (2)

James
James

Reputation: 2331

To follow this up, Cloud Dataproc 1.1, newly released this week, has Spark 2.0 and Scala 2.11 and is intended for production uses. See the release notes for more details!

Upvotes: 1

Patrick Clay
Patrick Clay

Reputation: 1349

Dataproc has a preview image version with Spark 2.0 (currently an earlier RC) and Scala 2.11, which you can try out today.

It is not supported for production use cases, because it is less vetted and we will push breaking changes as we refine the preview into a new fully supported image version. When it is fully stabilized we will ship a new image version with Spark 2 and Scala 2.11.

Upvotes: 1

Related Questions