CODEWITHSUNDEEP
python-3.xgoogle-cloud-platformpython-requestsgoogle-cloud-automl
TwinPenguins
TwinPenguins

Reputation: 495

Google AutoML Model Export using CLI/Python API

It has been a few days that GCP AutoML WebUI cannot export models for offline inference (support team says it is a front end component bug that has been fixed, it has been a few days and it not yet).

Said that, I am usually not a fan of GUI to do that, I was hoping I could use CLI or Python google-cloud-automl API to the job. I have to admit Google sucks at documentation, and they keep constantly changing functionalities without giving details and examples. Although all kind of functionalities are working (bucket listing, bucket creation, data upload to bucket, even model training etc.), but model export has been a headache and hard to configure. Before we proceed, please note that GOOGLE_APPLICATION_CREDENTIALS is added to the environment. What I have tried so far:

1. via POST:

# get the token
access_token = subprocess.check_output(
    'gcloud auth application-default print-access-token', shell=True)

data = {
  "outputConfig": {
    "modelFormat": "tf_saved_model", #or "tf-saved-model"
    "gcsDestination": {
      "outputUriPrefix": "gs://bucket/folder"
    }
  }
}


headers = {
    'Authorization': f'Bearer {access_token.decode().rstrip()}',
    'Content-Type': 'application/json; charset=utf-8',
}

url= f"https://automl.googleapis.com/v1/projects/{project_id}/locations/us-central1/models/{model_id}:export"

response = requests.post(url, headers=headers, json=data)

fails with the response.content:

b'{\n  "error": {\n    "code": 400,\n    "message": "List of found errors:\\t1.Field: name; Message: Required field is invalid\\t",\n    "status": "INVALID_ARGUMENT",\n    "details": [\n      {\n        "@type": "type.googleapis.com/google.rpc.BadRequest",\n        "fieldViolations": [\n          {\n            "field": "name",\n            "description": "Required field is invalid"\n          }\n        ]\n      }\n    ]\n  }\n}\n'

2. via google-cloud-automl

location = "us-central1"

client = automl.AutoMlClient()
model_full_id = client.model_path(project=project_id, location=location, model=model_id)

response = client.export_model(????)

There arguments are apparently required (if I run it client.export_model()):

1.Field: name; Message: Location ID was not provided.
2.Field: name; Message: Required field is invalid
3.Field: output_config.gcs_destination.output_uri_prefix; Message: Empty string was given for GCS path or prefix.

I am stuck right in the beginning how to pass arguments. I have tried to configure a metadata for location_to_be_written, and model_format via automl.types.ModelExportOutputConfig(???), but that is a trouble of its own.

Upvotes: 0

Views: 400

Answers (2)

Ksign
Ksign

Reputation: 817

  1. For the post method, You need to use the model-id (e.g. ICN124..., TBL543...)
    You can find yours on the UI => AutoML Models tab or by listing your models.
    I have managed to export a model by following the example you provided.

  2. For the library, you need to use the full path.
    Here is a working example:

from google.cloud import automl

project_id = "YOUR_PROJECT"
model_name="projects/YOUR_PROJECT_NUMBER/locations/us-central1/models/YOUR_MODEL_ID"
gcs_uri = "gs://bucket/folder"

client = automl.AutoMlClient()

gcs_destination = automl.GcsDestination(output_uri_prefix=gcs_uri)
output_config=automl.ModelExportOutputConfig(gcs_destination=gcs_destination, model_format="tflite")

request = automl.ExportModelRequest(name=model_name, output_config=output_config)

response = client.export_model(request=request)

BTW, the issue to export from the UI is now fixed and you should be able to export your models from it.

Upvotes: 1

philippatt
philippatt

Reputation: 1

I raised up the same questions. I have been using the WebPage to export the model very often before, and I was even able to export the model as of early last week. But not able to export any of the offline model since Friday :( Anyway, I tried your first Post Method, and it worked for me to export the model using the code.

Upvotes: 0

Related Questions