CODEWITHSUNDEEP
bert-language-modelhuggingface
ZL RONG
ZL RONG

Reputation: 51

Huggingface model inference issue

I'm trying to use my pre-trained huggingface model to predict.

    outputs = model(
        ids,
        mask,
        token_type_ids
    )
    outputs = torch.sigmoid(outputs.last_hidden_state).cpu().detach().numpy()
    return outputs[0][0]

The I got is

[[[0.5144298  0.68467325 0.4045368  ... 0.5698948  0.6843927  0.230076  ]
  [0.526383   0.6108195  0.46920577 ... 0.6635995  0.70778817 0.22947823]
  [0.47112644 0.6557672  0.49308282 ... 0.61219037 0.5811446  0.22059086]
  ...
  [0.46904904 0.66370267 0.4091996  ... 0.5381582  0.70973885 0.2500361 ]
  [0.47025025 0.6625398  0.40454543 ... 0.5423772  0.71071064 0.24768841]
  [0.47398427 0.658539   0.40038437 ... 0.53121835 0.7094869  0.2417065 ]]]

What I want is

[{'label': 'POSITIVE', 'score': 0.9998743534088135},
 {'label': 'NEGATIVE', 'score': 0.9996669292449951}]

Thanks ahead!!!

Upvotes: 2

Views: 134

Answers (1)

freek1
freek1

Reputation: 1

You are now performing a sigmoid transformation on all the values of the last hidden state of the bert (i presume?) model. Your softmax is not done over the correct dimension. The code below shows how to do it over the correct dimension (and using batches, which speeds up your code).

You ask for the label with the predicted probabilities (in a binary classification task), which can be gotten by:

    # other code
    logits_ls = [] # the list to store predicted probabilities for the class
    preds_ls = [] # the list to store the predicted classes
    for batch in loader: # assuming you use a dataloader
        logits = outputs.logits
        preds = torch.argmax(logits, dim=1)
        preds_ls.extend(preds.cpu().numpy())
        logits_ls.extend(torch.nn.functional.softmax(logits, dim=-1)[:, 1].cpu().numpy())
    return preds_ls, logits_ls

preds_ls will contain the labels. The logits_ls list will now contain the probabilities of predicting class 1 for each sample in the loader.

Getting your desired format can be done with simple manipulation of dictionaries in a list. E.g., for the positive predictions when preds_ls[i]==1, the score=logits_ls[i]. And when preds_ls[i]==0, the score=1-logits_ls[i].

EDIT: You did not mention your exact model, but be sure to use BertForSequenceClassification to have access to outputs.logits. Any other ...ForSequenceClassification model from Huggingface can do the same.

Upvotes: 0

Related Questions