CODEWITHSUNDEEP
pythonmachine-learningpytorchgradient-descentmlp
Bendaoud Simou
Bendaoud Simou

Reputation: 49

Pytorch mat1 and mat2 must have the same dtype mlp

So i am trying to do a function that trains an mlp using PyTorch. My code is as follows :

def mlp_gradient_descent(x,y , model , eta = 1e-6 , nb_iter = 30000)  : 

    loss_descent = []
    dtype = torch.float
    device = torch.device("cpu")
    x = torch.from_numpy(x)
    y = torch.from_numpy(y)

    params = model.parameters()

    learning_rate = eta
    for t in range(nb_iter):
        y_pred = model(x)
        loss = (y_pred - y).pow(2).sum()
        print(loss)
        if t % 100 == 99:
            print(t, loss.item())
            loss_descent.append([t, loss.item()])
        loss.backward()
        with torch.no_grad():
            for param in params : 
                param -= learning_rat*param.grad
            for param in params : 
                param = None

and i m having this error :

mat1 and mat2 must have the same dtype

Note that : The problem comes from the model(x) and x and y are numpy arrays.

Thank you all. And have a great day.

Upvotes: 2

Views: 3298

Answers (2)

Victor Björkgren
Victor Björkgren

Reputation: 45

To get the best performance, make sure to create your numpy arrays with dtype float32 from the start.

If that is not possible, change:

x = torch.from_numpy(x)
y = torch.from_numpy(y)

to:

x = torch.from_numpy(x).float()
y = torch.from_numpy(y).float()

Upvotes: 0

c_m_conlan
c_m_conlan

Reputation: 306

I faced a similar issue. The datatype of the weight matrix didn't match the datatype of the input vector.

In my case I defined the model as follows

class Multiclass(nn.Module):
def __init__(self, input_size, hidden_size1, output_size):
    super().__init__()
    self.hidden = nn.Linear(input_size, hidden_size1)
    self.act = nn.ReLU()
    self.output = nn.Linear(hidden_size1, output_size)
    
def forward(self, x):
    x = self.act(self.hidden(x))
    x = self.output(x)
    return x

And tested the datatype of the weight matrix using

print(model.hidden.weight.dtype)

They didn't match. So I had to explicitly define the datatype when defining my tensor, something like:

x_train_t = torch.tensor(X_hot_train, dtype=torch.float32)

Upvotes: 0

Related Questions