CODEWITHSUNDEEP
pythongoogle-cloud-platformpyspark
Jimmy K
Jimmy K

Reputation: 133

My google cloud cluster cannot find the module google in my python file?

I am trying to run a python file on the cloud, which runs fine locally. When I try to run it, I receive the below error:

ModuleNotFoundError: No module named 'google.cloud'

In my cluster script, I have installed tensorflow as below. Do I need to do something similar for Google.cloud?

CLUSTER = '{}-cluster'.format(PROJECT)
!gcloud dataproc clusters create $CLUSTER \
    --image-version 1.5-ubuntu18 --single-node \
    --master-machine-type n1-standard-8 \
    --master-boot-disk-type pd-ssd --master-boot-disk-size 100 \
    --max-idle 3600s \
    --initialization-actions=gs://goog-dataproc-initialization-actions-$REGION/python/pip-install.sh \
    --metadata=PIP_PACKAGES=tensorflow==2.4.0

Thanks

Upvotes: 0

Views: 116

Answers (1)

Fran Arenas
Fran Arenas

Reputation: 648

You need to install the google cloud library with pip install google-cloud or writing it in the metadata section (same as tensorflow)

Upvotes: 1

Related Questions