CODEWITHSUNDEEP
pythongoogle-cloud-dataflow
Silviu
Silviu

Reputation: 231

Can I use Dataflow for Python SDK from a Jupyter notebook?

I want to play with Dataflow for Python SDK from a Jupyter notebook. I am not sure what are the dependencies needed and if I can spread the code over multiple notebook cells or not. What are the steps involved?

Upvotes: 3

Views: 1109

Answers (1)

Silviu
Silviu

Reputation: 231

Yes! There are no special steps involved. For example, using a Conda environment (recommended for using IPython/Jupyter notebooks) the commands to start a Jupyter notebook are:

  1. conda create -n TESTENV jupyter
  2. source activate TESTENV
  3. pip install https://github.com/GoogleCloudPlatform/DataflowPythonSDK/archive/v0.2.3.tar.gz
  4. jupyter notebook

The commands above install version v0.2.3 of Python Dataflow. Please change it to the version desired. In the first notebook cell execute the following import statement:

import google.cloud.dataflow as df

Now you are all set. You can spread the workflow code over multiple cells. Check out the following notebook describing a very simple workflow: https://github.com/silviulica/WorkflowExamples/blob/master/notebooks/HelloWorld.ipynb

Upvotes: 5

Related Questions