CODEWITHSUNDEEP
machine-learningneural-networkdeep-learningpytorchrandom-seed
Matheus Ianzer
Matheus Ianzer

Reputation: 465

Does PyTorch seed affect dropout layers?

I came across the idea of seeding my neural network for reproducible results, and was wondering if pytorch seeding affects dropout layers and what is the proper way to seed my training/testing?

I'm reading the documentation here, and wondering if just placing these lines will be enough?

torch.manual_seed(1)
torch.cuda.manual_seed(1)

Upvotes: 3

Views: 5135

Answers (2)

Patrick J. Holt
Patrick J. Holt

Reputation: 696

Actually it depends on your device:

If cpu:

  • torch.manual_seed(1) == true.

If cuda:

  • torch.cuda.manual_seed(1)=true
  • torch.backends.cudnn.deterministic = True

Lastly, use the following code can make sure the results are reproducible among python, numpy and pytorch.

def setup_seed(seed):
    random.seed(seed)                          
    numpy.random.seed(seed)                       
    torch.manual_seed(seed)                    
    torch.cuda.manual_seed(seed)               
    torch.cuda.manual_seed_all(seed)           
    torch.backends.cudnn.deterministic = True  


setup_seed(42)

Upvotes: 1

Fábio Perez
Fábio Perez

Reputation: 26088

You can easily answer your question with some lines of code:

import torch
from torch import nn

dropout = nn.Dropout(0.5)
torch.manual_seed(9999)
a = dropout(torch.ones(1000))
torch.manual_seed(9999)
b = dropout(torch.ones(1000))
print(sum(abs(a - b)))
# > tensor(0.)

Yes, using manual_seed is enough.

Upvotes: 5

Related Questions