CODEWITHSUNDEEP
machine-learninggoogle-cloud-ml
anon_swe
anon_swe

Reputation: 9345

Google Cloud ML Engine: Apply Custom Function Before Training / Predicting

I currently pre-process some data on my laptop before passing it to ML Engine for training. Is it possible to apply a custom pre-processing function to my data and then train, all within ML Engine?

So instead of these steps:

  1. Pre-process data on laptop.

  2. Send pre-processed data to ML engine for training.

I would do:

  1. Define pre-processing function for ML Engine

  2. Send raw data to ML Engine, where it will:

    a) pre-process my data by applying the function I've specified and

    b) train on that data

Is this possible and, if so, how would I do it? I don't see anything in the docs.

Thanks!

Upvotes: 0

Views: 167

Answers (2)

rhaertel80
rhaertel80

Reputation: 8389

For preprocessing TensorFlow models, consider TensorFlow Transform (Getting Started Guide).

You may be interested in the chicago_taxi example, which includes a script for integrating the preprocessing with classification on Cloud ML Engine.

Upvotes: 1

gogasca
gogasca

Reputation: 10058

You can use some of the samples code here:

Pre-processing is done using DataFlow and then training in ML Engine using the output generated during the pre-process phase.

Upvotes: 1

Related Questions