CODEWITHSUNDEEP
pythontensorflowkerastensorflow2.0keras-layer
Ronen
Ronen

Reputation: 45

Keras Tensorflow custom layer only called once

After two days of intense searching, I did not find answers in keras docs/stack of.

I'm trying to create a custom layer to perform augmentation on each batch at some probability (now 1 for demonstration). the data is an image with one channel.

I need a way to know if the augmentation actually happens while training because the prints inside the augmentation functions only happen once when the network is defined, and another when the fit function is activated.

any help will be much appreciated, Thanks!

Basic runnable working example, notice the number of prints from call() :

from tensorflow.keras.layers import Input, Dense, Conv2D, Flatten
from tensorflow.keras.models import Sequential
import random
import numpy as np
import tensorflow as tf
from tensorflow.keras import optimizers


# Custom Augmentation layer
class AugmentationLayer(tf.keras.layers.Layer):
    def __init__(self, p):
        super(AugmentationLayer, self).__init__()
        self.p = p

    def call(self, inputs, training=None):
        if not training:
            return inputs
        print('Enter Custom layer call, training = ', training)
        
        if random.random() < self.p:  
            inputs *= 2
        return inputs

# Training Script

# parameters
height = 38
width = 22
numClasses = 2
ydata = np.array([[1, 0], [1, 0]])   # hot-encoded labels

# create 2 images for network training data
Xdata = []
for i in range(2):
    xx = np.random.uniform(-1, 1, (height, width))
    Xdata.append(xx)

# create array from list
Xdata = np.array(Xdata)

# make dims n_samples X height X width X channels = (2, 38, 22, 1)
Xdata = np.reshape(Xdata, (Xdata.shape[0], Xdata.shape[1], Xdata.shape[2], 1))

model = Sequential()
model.add(Input(shape=(height, width, 1)))
model.add(AugmentationLayer(1))                   # Added custom layer
model.add(Conv2D(32, (2, 2), activation='relu'))
model.add(Flatten())
model.add(Dense(numClasses, activation='softmax'))

model.compile(
    loss='categorical_crossentropy',
    metrics=['accuracy'],
    optimizer=optimizers.Adam(learning_rate=0.001))

model.summary()

model.fit(
    Xdata,
    ydata,
    epochs=10,
    batch_size=2,
    verbose=1)

Upvotes: 0

Views: 347

Answers (2)

Ronen
Ronen

Reputation: 45

for someone in the future, i will add that when compiling the model, there is a default value that prevents certain function to run from inside the custom layers (that's why regular print() did not work), for performance purposes. amongst them are: print(), np.save() and probably more.

model.compile(..., run_eagerly=False)   # default

if you want to enable all the function mentioned above, compile your model with:

model.compile(..., run_eagerly=True)

Upvotes: 0

Leo
Leo

Reputation: 54

If you use tf.print() it will print when running in graph execution as well. You can read more about it here: https://www.tensorflow.org/api_docs/python/tf/print.

Upvotes: 0

Related Questions