CODEWITHSUNDEEP
pytorchconv-neural-network
Wasi Ahmad
Wasi Ahmad

Reputation: 37761

Convolutional NN for text input in PyTorch

I am trying to implement a text classification model using a CNN. As far as I know, for text data, we should use 1d Convolutions. I saw an example in pytorch using Conv2d but I want to know how can I apply Conv1d for text? Or, it is actually not possible?

Here is my model scenario:

Number of in-channels: 1, Number of out-channels: 128 
Kernel size : 3 (only want to consider trigrams)
Batch size : 16

So, I will provide tensors of shape, <16, 1, 28, 300> where 28 is the length of a sentence. I want to use Conv1d which will give me 128 feature maps of length 26 (as I am considering trigrams).

I am not sure, how to define nn.Conv1d() for this setting. I can use Conv2d but want to know is it possible to achieve the same using Conv1d?

Upvotes: 12

Views: 7739

Answers (2)

Wasi Ahmad
Wasi Ahmad

Reputation: 37761

This example of Conv1d and Pool1d layers into an RNN resolved my issue.

So, I need to consider the embedding dimension as the number of in-channels while using nn.Conv1d as follows.

m = nn.Conv1d(200, 10, 2) # in-channels = 200, out-channels = 10
input = Variable(torch.randn(10, 200, 5)) # 200 = embedding dim, 5 = seq length
feature_maps = m(input)
print(feature_maps.size()) # feature_maps size = 10,10,4

Upvotes: 16

Roger Trullo
Roger Trullo

Reputation: 1584

Although I don't work with text data, the input tensor in its current form would only work using conv2d. One possible way to use conv1d would be to concatenate the embeddings in a tensor of shape e.g. <16,1,28*300>. You can reshape the input with view In pytorch.

Upvotes: 2

Related Questions