CODEWITHSUNDEEP
pythonfunctionmodelbert-language-model
Shorouk Adel
Shorouk Adel

Reputation: 157

output from bert gives str not tensor in forward function

class BertModel(nn.Module):
    def __init__(self,pre_trained='bert-base-uncased'):
        super().__init__()        
        self.bert = AutoModel.from_pretrained(pre_trained)
        self.dropout = nn.Dropout(0.1)
        self.relu =  nn.ReLU()
        self.fc1 = nn.Linear(768,512)
        self.fc2 = nn.Linear(512,6)
      
    
        
    def forward(self,inputs, mask, labels):
        
        pooled, cls_hs = self.bert(input_ids=inputs,attention_mask=mask)
        print(pooled)
        print(cls_hs)  
        print(inputs) 
        print(mask)   
        x = self.fc1(cls_hs)
        print(1) 
        x = self.relu(x)
        print(2) 
        x = self.dropout(x)
        print(3) 
      # output layer
        x = self.fc2(x)
        print(4)
      # apply softmax activation
        x = self.softmax(x)
        print(5)

last_hidden_state pooler_output

tensor([[ 101, 2342, 2393, ..., 0, 0, 0], [ 101, 14477, 4779, ..., 4839, 6513, 102], [ 101, 14777, 2111, ..., 13677, 3613, 102], ..., [ 101, 2113, 14047, ..., 0, 0, 0], [ 101, 5683, 3008, ..., 0, 0, 0], [ 101, 19046, 2075, ..., 2050, 3308, 102]]) tensor([[1, 1, 1, ..., 0, 0, 0], [1, 1, 1, ..., 1, 1, 1], [1, 1, 1, ..., 1, 1, 1], ..., [1, 1, 1, ..., 0, 0, 0], [1, 1, 1, ..., 0, 0, 0], [1, 1, 1, ..., 1, 1, 1]])

in linear(input, weight, bias) if has_torch_function_variadic(input, weight, bias): return handle_torch_function(linear, (input, weight, bias), input, weight, bias=bias) return torch._C._nn.linear(input, weight, bias)

TypeError: linear(): argument 'input' (position 1) must be Tensor, not str

pooled, cls_hs printed as string last_hidden_state, pooler_output tensor with out any tensor

Upvotes: 0

Views: 205

Answers (1)

Mehmet Calikus
Mehmet Calikus

Reputation: 123

Try to replace the line where you downloaded the pretrained bert model with:

self.model = AutoModel.from_pretrained(pre_trained, return_dict=False)

Upvotes: 1

Related Questions