CODEWITHSUNDEEP
tensorflowgoogle-cloud-platformpredictgoogle-cloud-ml
Stav1
Stav1

Reputation: 130

Google Cloud ml-engine fails predicting multiple inputs

Predictions only successful when providing a single instance instance.json.

Test 1: Contents of instance.json:

{"serving_input": [20.0, 0.0, 1.0 ... 0.16474569041197143, 0.04138248072194471], "prediction_id": 0, "keep_prob": 1.0}

Prediction (same output for local and online prediction)

gcloud ml-engine local predict --model-dir=./model_dir --json-instances=instances.json

Output:

SERVING_OUTPUT                         ARGMAX  PREDICTION_ID  SCORES      TOP_K
[-340.6920166015625, -1153.0877685546875]  0       0              [1.0, 0.0]  [1.0, 0.0]

Test 2: Contents of instance.json:

{"serving_input": [20.0, 0.0, 1.0 ... 0.16474569041197143, 0.04138248072194471], "prediction_id": 0, "keep_prob": 1.0}
{"serving_input": [21.0, 2.0, 3.0 ... 3.14159265359, 0.04138248072194471], "prediction_id": 1, "keep_prob": 1.0}

Output:

.. Incompatible shapes: [2] vs. [2,108] .. (_arg_keep_prob_0_1, Model/dropout/random_uniform)

Where as 108 is the size of the first hidden layer(net_dim=[2015,108,2]). (Initialized with tf.nn.dropout, thus the keep_prob=1.0)

Exporting code:

probabilities = tf.nn.softmax(self.out_layer)
top_k, _ = tf.nn.top_k(probabilities, self.network_dim[-1])

prediction_signature = (
    tf.saved_model.signature_def_utils.predict_signature_def(
        inputs={'serving_input': self.x, 'keep_prob': self.keep_prob,
                'prediction_id': self.prediction_id_in},
        outputs={'serving_output': self.out_layer, 'argmax': tf.argmax(self.out_layer, 1),
                 'prediction_id': self.prediction_id_out, 'scores': probabilities, 'top_k': top_k}))

builder.add_meta_graph_and_variables(
    sess,
    tags=[tf.saved_model.tag_constants.SERVING],
    signature_def_map={
        tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
            prediction_signature
    },
    main_op=tf.saved_model.main_op.main_op())
builder.save()

How can i format the instance.json to perform a batched prediction? (Prediction with multiple input instances)

Upvotes: 0

Views: 168

Answers (1)

Lak
Lak

Reputation: 4166

The problem is not in the JSON. Check to see how you are using self.x I think that your code is assuming that it's a 1D array, when you should treat it as a tensor of shape [?, 108]

Upvotes: 1

Related Questions