CODEWITHSUNDEEP
pytorch
Samrat Hasan
Samrat Hasan

Reputation: 383

When should I use nn.ModuleList and when should I use nn.Sequential?

I am new to Pytorch and one thing that I don't quite understand is the usage of nn.ModuleList and nn.Sequential. Can I know when I should use one over the other? Thanks.

Upvotes: 36

Views: 15467

Answers (2)

jdhao
jdhao

Reputation: 28329

Both nn.ModuleList and nn.Sequential are containers that contains pytorch nn modules.

nn.ModuleList just stores a list nn.Modules and it does not have a forward() method. So you can not call it like a normal module.

On the other hand, nn.Sequential also contains a list of modules, but you need to make sure that output from current module can be fed into its next module, otherwise, you will get an error. You can think nn.Sequential as a module and you can call it with a input like the normal module.

You can define a ModuleList, but you can not call this mlist with input, this will cause an error:

import torch
import torch.nn as nn

mlist = nn.ModuleList([nn.Linear(10, 10) for i in range(5)])
x = torch.rand(64, 10)

out = mlist(x)  # this will cause an error

For nn.Sequential, this is not the case:

seqlist = nn.Sequential(nn.Linear(10, 10), nn.ReLU(), nn.Linear(10, 20))

x = torch.random(64, 10)
out = seqlist(x)  # runs fine, shape of out: (64, 20)

Ref:

Upvotes: 4

Egor Lakomkin
Egor Lakomkin

Reputation: 1434

nn.ModuleList does not have a forward method, but nn.Sequential does have one. So you can wrap several modules in nn.Sequential and run it on the input.

nn.ModuleList is just a Python list (though it's useful since the parameters can be discovered and trained via an optimizer). While nn.Sequential is a module that sequentially runs the component on the input.

Upvotes: 41

Related Questions