CODEWITHSUNDEEP
machine-learningneural-networkpytorch
DOT
DOT

Reputation: 329

Is this a right way to train and test the model using Pytorch?

I am trying to implement a binary classification with Neural Networks with Pytorch to train a model as shown below:

model = nn.Sequential(
    bnn.BayesLinear(prior_mu=0, prior_sigma=0.1, in_features=196, out_features=300),
    nn.ReLU(),
    bnn.BayesLinear(prior_mu=0, prior_sigma=0.1, in_features=300, out_features=196),
)
cross_entropy_loss = nn.CrossEntropyLoss()
klloss = bnn.BKLLoss(reduction='mean', last_layer_only=False)
klweight = 0.01
optimizer = optim.Adam(model.parameters(), lr=0.01)

Traing part:

 #training
    for step in range(200):
        models = model(data_tensor)
        cross_entropy = cross_entropy_loss(models, target_tensor)
        #cross_entropy = 0
        kl = klloss(model)
        total_cost = cross_entropy + klweight*kl
    
        optimizer.zero_grad()
        total_cost.backward()
        optimizer.step()
      
    _, predicted = torch.max(models.data, 1)
    final = target_tensor.size(0)
    correct = (predicted == target_tensor).sum()
    print('- Accuracy: %f %%' % (100 * float(correct) / final))
    print('- CE : %2.2f, KL : %2.2f' % (cross_entropy.item(), kl.item()))

Question 1: Is it a right way to train a model? In many articles I found that there is a section to iterate over the DataLoader for training data, such as :

for i, data in enumerate(trainloader, 0):
 # Get inputs
 inputs, targets = data

Question 2: What is the use of this loop if I can directly give the data features(inputs) as data_tensor and data labels(target) as target_tensor? Because when iterating through data loader, it takes more time.

I am not sure how to test my model. I did as given below:

correct = 0
total = 0
with torch.no_grad():
    for step in range(data_tensor_test.size(0)):
        models = model(data_tensor_test)
        _, predicted = torch.max(models.data, 1)
        total += target_tensor_test.size(0)
        correct += (predicted == target_tensor_test).sum().item()

print('Accuracy of the test data: %d %%' % (
    100 * correct / total))

Question 3: Is it right way to test the model that I trained above?

Upvotes: 0

Views: 956

Answers (2)

Tomasz Kaczmarski
Tomasz Kaczmarski

Reputation: 136

  1. your model is bayesian classification not binary
  2. seems your ce missing weights see tutorial below https://github.com/Harry24k/bayesian-neural-network-pytorch/blob/master/demos/Custom%20KL%20loss%20with%20Iris%20Data.ipynb

Upvotes: 1

jhso
jhso

Reputation: 3283

I'll attempt to answer your questions:

Question 1 & 2: Is it a right way to train a model? In many articles I found that there is a section to iterate over the DataLoader for training data.

You should use a dataset with a dataloader when training in pytorch for several reasons:

  1. It allows us to sample our data randomly
  2. It doesn't preload data into memory, which is particularly useful for huge datasets
  3. It operates in the background of code so fetches data parallel to training, thus saving time
  4. It's very efficient at batching your data

What you're doing here is seemingly running your model on every element in your data at once. If you only have 32 points in your data this might be fine (although really not optimal because you have such limited data) but there is a balance to strike between running your optimizer and exposing your model to learning opportunities.

I would guess this takes longer because your model is very small and it probably takes longer to run the data fetching than it does when it's already preloaded in memory. It is hard to answer this without knowing the size of your dataset and the batch size of your processing.

Question 3: Is it right way to test the model that I trained above? You need to set your model to its evaluation stage using model.eval() before you run any inference code. I also don't understand the point of your for loop since you just pass through the same data every time. I would generally run something like this:

correct = 0
total = 0
with torch.no_grad():
    model.eval()
    for step,(dat,lab) in enumerate(dataloader_test):
        models = model(dat)
        _, predicted = torch.max(models.data, 1)
        total += dat.size(0)
        correct += (predicted == lab).sum().item()

print('Accuracy of the test data: %d %%' % (
    100 * correct / total))

Upvotes: 2

Related Questions