CODEWITHSUNDEEP
kerasconvolution
zhf061
zhf061

Reputation: 321

keras: 1D convolutions with different filter for each sample in mini-batch

Generally, we do 1D convolution operation over the sampels of a mini-batch with the same set of filters. But now I would like to convolve with different filter for each sample in the mini-batch. Is there any way I could do that in keras, especially if the mini-batch size is not known?

Concretely, I have input data with the shape of (batch_size, maxlen, input_dim), and I have generated a set of filters of the form (batch_size, output_dim, kernel_size, input_dim). Any ideas I could convolve the input with the set of filters?

Upvotes: 1

Views: 340

Answers (1)

Daniel Möller
Daniel Möller

Reputation: 86620

This is very tricky and we're getting help from K.depthwise_conv2d (the only convolution that treats channels individually), where we will transform the samples into channels, have the desired outputs per channel and then reshape back to the expected

So, the idea is:

1 - Transform input shape (with proper reordering)

#from `(batch_size, maxlen, input_dim)` 
#to `(1, maxlen, input_dim, batch_size)`   
x = K.expand_dims(x, axis=0)
x = K.permute_dimensions(x, (0,2,3,1))

2 - Have filters with shape (kernel_size, input_dim, batch_size, output_dim) 

#transform your kernels:
filters = K.permute_dimensions(filters, (2, 3, 0, 1))

3 Transform the results (with proper reordering):

#from `(1, result_len, 1, batch_size * output_dim)` 
#to `(batch_size, result_len, output_dim)`   
results = K.reshape(results, (output_length, -1, output_dim) #-1 for batch size
results = K.permute_dimensions(results, (1,0,2))

Here is a sample code:

from keras.layers import *
import keras.backend as K
from keras.models import Model

#dimensions
length = 11 #maxlen
features = 3 #input_dim
filtersize = 2 #kernel_size
out_dim = 5 #output_dim
samples = 7 #batch_size

#keep track of output length for reshaping
outlen = length - filtersize + 1

#creating dummy filters with the desired shape
npfilters = np.arange(features*filtersize*out_dim*samples)
npfilters = npfilters.astype(np.float64)
npfilters = npfilters.reshape((filtersize, features, samples, out_dim))
kerasfilters = K.variable(npfilters)

#function that performs the convolution
def sample_wise_conv1d(x):

    #reshape and reorder inputs properly
    x= K.expand_dims(x, axis=0)
    x = K.permute_dimensions(x, (0,2,3,1))
    print('in shape', K.int_shape(x))

    #perform the convolution
    print("filter shape", K.int_shape(kerasfilters))
    results =  K.depthwise_conv2d(x, kerasfilters)
    print('out shape', K.int_shape(results))

    #reshape and reorder the results properly
    results = K.reshape(results, (outlen, samples, out_dim))
    results = K.permute_dimensions(results, (1,0,2))
    print('final shape', K.int_shape(results))

    return results


#creating a model that performs the operation
inputs = Input((length, features))
outputs = Lambda(sample_wise_conv1d)(inputs)

model = Model(inputs, outputs)

#predicting from the model
inputdata = np.arange(samples*length*features).reshape((samples, length, features))
results = model.predict(inputdata)

print(results.shape)

Testing the code

#creating a single conv1D model for each sample
for i in range(samples):

    #get the respective input sample and filter
    x = inputdata[i:i+1]
    filts = npfilters[:,:,i,:]
    print(filts.shape)

    #make a model with conv1d
    in1D = Input((length, features))
    out1D = Lambda(lambda x: K.conv1d(x, K.variable(filts)))(in1D)
    model1D = Model(in1D, out1D)

    #compare this model's predictions with the respective prediction from the function
    pred1D = model1D.predict(x)
    pred2D = results[i]

    print(pred1D == pred2D)

Upvotes: 1

Related Questions