CODEWITHSUNDEEP
pythontensorflowdeep-learningtensorflow-estimator
saru
saru

Reputation: 255

Custom input function for estimator instead of tf.data.dataset

I want to know if anyone has created their own custom input function for tensorflow's estimator ? like in (link) this image:enter image description here

where the say it is recommended to use tf.data.dataset. But I do not want to use that one, as I want to write my own iterator which yields data in batches and shuffles it as well. 

    def data_in(train_data):
      data = next(train_data)
      ff = list(data)
      tf.enable_eager_execution()
      imgs = tf.stack([tf.convert_to_tensor(np.reshape(f[0], [img_size[0], img_size[1], img_size[2]])) for f
                                        in ff])
      lbls = tf.stack([f[1] for f in ff])
      print('TRAIN data: %s %s ' % (imgs.get_shape(), lbls.get_shape()))
      return imgs, lbls

output: TRAIN data: (10, 32, 32, 3) (10,)

where train_data is a generator object which iterates through my dataset using iter and np.reshape(f[0], [img_size[0], img_size2, img_size2] basically reshapes the data I extract to the required dimensions and it is a batch of an entire dataset. I use stack to convert the list of tensors to convert to stacked tensors. But when I use this with estimators I get an error for the features provided to the model saying the features do not have get_shape(). When I test it without an estimator it works well and it get_shape() also works well.

Upvotes: 0

Views: 396

Answers (1)

saru
saru

Reputation: 255

Hey kvish I figured it out how to do it. I just had to add these lines

experiment = tf.contrib.learn.Experiment(
cifar_classifier,
train_input_fn=lambda: data_in(),
eval_input_fn=lambda: data_in_eval(),
train_steps=train_steps)

I know experiment is deprecated, I will also do it with estimator now :)

Upvotes: 1

Related Questions