CODEWITHSUNDEEP
pythontensorflowmachine-learningkerascomputer-vision
explr_1298
explr_1298

Reputation: 43

why is SeparableConv2D slower than Conv2D?

I've been experimenting with different types of convolution layers, to check their computational speeds. my code, in the beginning, was as follows.

def conv_block_A(layer):
    block = tf.keras.layers.Conv2D(filters=128, kernel_size=3, strides=1, padding='same')(layer)
    block = tf.keras.layers.Conv2D(filters=196, kernel_size=3, strides=1, padding='same')(block)
    block = tf.keras.layers.Conv2D(filters=128, kernel_size=3, strides=1, padding='same')(block)
    block = tf.keras.layers.BatchNormalization(momentum=0.8)(block)
    block = tf.keras.layers.LeakyReLU(alpha=0.2)(block)

    return block

after going through a few blogs, I changed my code as 

def conv_block_A(layer):
    block = tf.keras.layers.SeparableConv2D(filters=128, kernel_size=3, strides=1, padding='same')(layer)
    block = tf.keras.layers.SeparableConv2D(filters=196, kernel_size=3, strides=1, padding='same')(block)
    block = tf.keras.layers.SeparableConv2D(filters=128, kernel_size=3, strides=1, padding='same')(block)
    block = tf.keras.layers.BatchNormalization(momentum=0.8)(block)
    block = tf.keras.layers.LeakyReLU(alpha=0.2)(block)

    return block

the training process became twice as fast on CPUs, but, the training has become very slow on Tesla T4. what could be the reason?

Upvotes: 3

Views: 1752

Answers (1)

cosine
cosine

Reputation: 21

It's a known issue with GPU's it was fixed in #33836. Also, you should update your GPU driver. As a rule of thumbs speedup by doing separable convolution is more noticeable with large kernel sizes because of the overhead involving doing two convolutions might be larger than the speedups.

Upvotes: 1

Related Questions