CODEWITHSUNDEEP
pytorchgcloudgoogle-cloud-ml
user6377061
user6377061

Reputation: 65

Is there any way to use both a GPU accelerator and Torch in google cloud AI platform for model deployment?

I already have a torch model (BERT), and I'd like to use the ai-platform service to get online predictions using a GPU, but I can't figure out how to do it.

The following command, without an accelerator, works:

gcloud alpha ai-platform versions create {VERSION} --model {MODEL_NAME} --origin=gs://{BUCKET}/models/ --python-version=3.5 --runtime-version=1.14 --package-uris=gs://{BUCKET}/packages/my-torch-package-0.1.tar.gz,gs://cloud-ai-pytorch/torch-1.0.0-cp35-cp35m-linux_x86_64.whl --machine-type=mls1-c4-m4 --prediction-class=predictor.CustomModelPrediction

However, if I try to add the accelerator parameter:

--accelerator=^:^count=1:type=nvidia-tesla-k80

I get the following error message:

ERROR: (gcloud.alpha.ai-platform.versions.create) INVALID_ARGUMENT: Field: version.machine_type Error: GPU accelerators are not supported on the requested machine type: mls1-c4-m4
- '@type': type.googleapis.com/google.rpc.BadRequest
  fieldViolations:
  - description: 'GPU accelerators are not supported on the requested machine type:
      mls1-c4-m4'
    field: version.machine_type

But if I use a different machine type, that I know I can use with an accelerator, I get the following error:

ERROR: (gcloud.alpha.ai-platform.versions.create) FAILED_PRECONDITION: Field: framework Error: Machine type n1-highcpu-4 does not support CUSTOM_CLASS.
- '@type': type.googleapis.com/google.rpc.BadRequest
  fieldViolations:
  - description: Machine type n1-highcpu-4 does not support CUSTOM_CLASS.
    field: framework

It's like any machine that supports GPU accelerators doesn't support custom classes (required AFAIK to use Torch), and any machine that supports custom classes doesn't support GPU accelerators.

Any way to make it work?

There are a bunch of tutorials on how to use ai-platform with Torch, but I can't see the point of using gcloud to train and predict if you have to do everything on the CPU so that feels very odd to me.

Upvotes: 1

Views: 330

Answers (2)

gogasca
gogasca

Reputation: 10058

Pytorch + GPU is not available in AI Platform Prediction, but you can use Deep Learning VM images and create your custom Pytorch serving with GPU

Update: You can use now AI Platform Prediction with containers.

Upvotes: 1

Albert Albesa
Albert Albesa

Reputation: 417

As for now, using Custom Prediction Routines is in Beta. In addition, using other machine types than mls1-c1-m2 is also in Beta.

Nevertheless, as you can see in the previously referenced link, GPU's are not available for mls1-like machines. At the same time, these are the only machine types that allow models outside TensorFlow.

In summary, probably deploying your prediction model in Torch and using a GPU might not be a feasible option right now.

Upvotes: 1

Related Questions