How to use random zoom in keras tensorflow 2.3

Question

I am trying to add random zoom to my images that are tiff files with 128x160 resolution 1 channel but new version of random zoom for keras tensorflow has gotten me confused, i don't understand how should be the tuple format it expects as the zoom range arguement.

From the documentation.

tf.keras.preprocessing.image.random_zoom(
    x, zoom_range, row_axis=1, col_axis=2, channel_axis=0, fill_mode='nearest',
    cval=0.0, interpolation_order=1
)

I need to add some random zoom to my image, and i am trying like this:

zoom_range = ((0.4, 0.4))
img = tf.keras.preprocessing.image.random_zoom(
    img, zoom_range, row_axis=1, col_axis=2, channel_axis=0, fill_mode='nearest',
    cval=0.0, interpolation_order=1
)

The output is:

TypeError: float() argument must be a string or a number, not 'NoneType'

How exactly should i pass as parameter any random zoom amount to my images?

Public kaggle notebook here:

https://www.kaggle.com/puelon/notebook75c416766a

TypeError: in user code:

    <ipython-input-4-9ba0455797a4>:17 load  *
        img = tf.keras.preprocessing.image.random_zoom(img, zoom_range, row_axis=0, col_axis=1, channel_axis=2, fill_mode='nearest')
    /opt/conda/lib/python3.7/site-packages/keras_preprocessing/image/affine_transformations.py:153 random_zoom  *
        x = apply_affine_transform(x, zx=zx, zy=zy, channel_axis=channel_axis,
    /opt/conda/lib/python3.7/site-packages/keras_preprocessing/image/affine_transformations.py:321 apply_affine_transform  *
        transform_matrix = transform_matrix_offset_center(
    /opt/conda/lib/python3.7/site-packages/keras_preprocessing/image/affine_transformations.py:246 transform_matrix_offset_center  *
        o_x = float(x) / 2 + 0.5
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/autograph/operators/py_builtins.py:195 float_  **
        return _py_float(x)
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/autograph/operators/py_builtins.py:206 _py_float
        return float(x)
    
    TypeError: float() argument must be a string or a number, not 'NoneTyp

e'

TypeError                                 Traceback (most recent call last)
<ipython-input-4-9ba0455797a4> in <module>
     27     train1, train2, test1 = d
     28     train_ds = tf.data.Dataset.from_tensor_slices(train1 + train2).\
---> 29         shuffle(len(train1) + len(train2)).map(load).batch(4)
     30     test_ds = tf.data.Dataset.from_tensor_slices(test1).\
     31         shuffle(len(test1)).map(load).batch(4)



for i in range(len(groups)):
    d = deque(groups)
    d.rotate(i)
    train1, train2, test1 = d
    train_ds = tf.data.Dataset.from_tensor_slices(train1 + train2).\
        shuffle(len(train1) + len(train2)).map(load).batch(4)
    test_ds = tf.data.Dataset.from_tensor_slices(test1).\
        shuffle(len(test1)).map(load).batch(4)

Answer 1

Probably your img is of wrong object type. For random_zoom(...) function you need to provide input as tensor or 3D numpy array of shape (height, width, channels) i.e. for RGB image of size 300x200 array should be of shape (200, 300, 3). This kind of numpy array can be obtained for example by PIL library like in code below.

Also if you're having TF code then you're dealing with tensors, but random_zoom needs to know all dimensions, their integer sizes. Tensors may have None size for some dimensions if it is unknown at graph construction time, and probably this is causing the error about NoneType in your case. To overcome this you need to wrap random_zoom usage into numpy function interface, this will force function input to be numpy array instead of tensor and numpy arrays always have all dimensions with known sizes. I've implemented this wrapping in my code down below too.

Also you probably need to change row_axis=1, col_axis=2, channel_axis=0 to row_axis=0, col_axis=1, channel_axis=2 because channels (colors) go usually in least significant dimension (last).

Documentation for tf.keras.preprocessing.image.random_zoom.

I've implemented simple code next that works.

Input in code looks like this:

output looks like this:

Next code can be also run here online.

# Needs: python -m pip install tensorflow numpy pillow requests
import tensorflow as tf, numpy as np, PIL.Image, requests, io

tf.compat.v1.enable_eager_execution()

zoom_range = (0.4, 0.5)

img = PIL.Image.open(io.BytesIO(requests.get('https://i.sstatic.net/Fc3Jb.png').content))
#img = PIL.Image.open('Ruler-Big-Icon-PNG.png')
img = np.array(img)

img = tf.convert_to_tensor(img) # This line is not needed if you already have a tensor.

# You need only this single line of code to fix your issue!
img = tf.numpy_function(lambda img: tf.keras.preprocessing.image.random_zoom(
    img, zoom_range, row_axis=0, col_axis=1, channel_axis=2, fill_mode='nearest',
), [img], tf.float32)

img = np.array(img) # This line is not needed if you plan img to be a tensor futher

# Example output is https://i.sstatic.net/MWk9T.png
PIL.Image.fromarray(img).save('result.png')

Answer 2

I'm generally not convinced that using Keras preprocessing functions outside of where they belong is the right approach. A simple way would be to use tf.image.random_crop. Assuming your image is larger than (200, 200, 3), you can just use this line:

img = tf.image.random_crop(img, (200, 200, 3))

Let's try an example. Original image:

import tensorflow as tf
import skimage
import matplotlib.pyplot as plt
import numpy as np

X = np.stack([skimage.data.chelsea() for _ in range(10)])

ds = tf.data.Dataset.from_tensor_slices(X).\
    map(lambda x: tf.image.random_crop(x, (200, 200, 3)))

plt.imshow(next(iter(ds)))
plt.show()

Randomly cropped image of size (200, 200, 3):