CODEWITHSUNDEEP
apache-sparkibm-cloud
Gaurav
Gaurav

Reputation: 31

Bluemix Spark Service

Firstly, I need to admit that I am new to Bluemix and Spark. I just want to try out my hands with Bluemix Spark service.

I want to perform a batch operation over, say, a billion records in a text file, then I want to process these records with my own set of Java APIs.

This is where I want to use the Spark service to enable faster processing of the dataset.

Here are my questions:

  1. Can I call Java code from Python? As I understand it, presently only Python boilerplate is supported? There are few a pieces of JNI as well beneath my Java API.

  2. Can I perform the batch operation with the Bluemix Spark service or it is just for interactive purposes?

  3. Can I create something like a pipeline (output of one stage goes to another) with Bluemix, do I need to code for it ?

I will appreciate any and all help coming my way with respect to above queries.

Look forward to some expert advice here.

Thanks.

Upvotes: 3

Views: 291

Answers (1)

Anup Kumar
Anup Kumar

Reputation: 11

The IBM Analytics for Apache Spark sevices is now available and it allow you to submit a java code/batch program with spark-submit along with notebook interface for both python/scala.

Earlier, the beta code was limited to notebook interactive interface.

Regards Anup

Upvotes: 1

Related Questions