CODEWITHSUNDEEP
tensorflowgoogle-cloud-platformgoogle-cloud-mlgoogle-cloud-datalab
Nicky Feller
Nicky Feller

Reputation: 3889

Google Cloud: How to use Cloud ML in Cloud Datalab

I am training models in keras and tensorflow in Google Cloud Datalab. I want to run the training jobs in Cloud ML. Is it possible to do this with a %% bash [...] command?

Let's say my model looks something like this:

X = tf.placeholder(tf.float32, shape=[None, num_inputs])
hidden = fully_connected(X, num_hidden, activation_fn=None)
outputs = fully_connected(hidden, num_outputs, activation_fn=None)
loss = tf.reduce_mean(tf.square(outputs - X))
optimizer = tf.train.AdamOptimizer(learning_rate)
train  = optimizer.minimize( loss)
init = tf.global_variables_initializer()

num_steps = 3000
    with tf.Session() as sess:
    sess.run(init)

    for iteration in range(num_steps):
        sess.run(train,feed_dict={X: scaled_data})

How can I go about training this in a cloud ML job?

Upvotes: 0

Views: 378

Answers (1)

Bradley Jiang
Bradley Jiang

Reputation: 424

You can with the following steps:

  1. Organize your code (https://cloud.google.com/ml-engine/docs/tensorflow/packaging-trainer) so it looks like: ./ ./setup.py ./train/ ./init.py ./task.py

    Here setup.py includes your dependencies. task.py is the python module for your trainer so your code should move there. Also here is a CloudML sample: https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/iris/trainer

    In Datalab or Jupyter, there is a %%file command to save files to local disk. This is one way to create and edit your py files.

  2. Run "gcloud ml-engine jobs submit training" command to submit a training request. For example:

    gcloud ml-engine jobs submit training my-job-id --runtime-version=1.6 --module-name trainer.task --job-dir=gs://mybucket/mydir --region us-central1 -- --param1 value1 --param2 value2

Upvotes: 2

Related Questions