CODEWITHSUNDEEP
pythontensorflowdeep-learningneural-networkpytorch
Aioku Takume
Aioku Takume

Reputation: 1

How to build neural network in this structure？with different nodes connects to different number of nodes in next layer

I only know how to use the built-in network like RNN of LSTM in PyTorch. But they tend to deal with every node in the previous layer that will give information to all nodes in the next layer.

I want to do something different but don't know how to code it myself. Like in this figure: the node a maps to all [d, e, f] three nodes in layer 2, while node b maps to [e,f] and node c only maps to [f]. As a result, node d will only contain information from a, while e will contain information from [a, b]. And f will contain information from all nodes in the previous layer. Does anyone know how to code this structure? PLS give me some insight I'll be very grateful :D

Structure

Upvotes: 0

Views: 116

Answers (1)

ayandas
ayandas

Reputation: 2288

When you have a layer that looks like Fully-Connected layer but with custom connectivity, use a mask with proper structure.

Let's say x = [a, b, c] is your 3-dim input and W denotes the connectivity matrix.

>> x
tensor([[0.1825],
        [0.9598],
        [0.2871]])
>> W
tensor([[0.7459, 0.4669, 0.9687],
        [0.9016, 0.4690, 0.0471],
        [0.5926, 0.9700, 0.5222]])

then W[i][j] points to the connecting weight between jth input and ith output neuron. To build the structure similar to your toy example, we would make a mask like this

>> mask
tensor([[1., 0., 0.],
        [1., 1., 0.],
        [1., 1., 1.]])

Then you can simply mask the W

>> (mask * W) @ x
tensor([[0.1361],
        [0.6147],
        [1.1892]])

Note: @ is matrix multiplication and * is pointwise multiplication.

Upvotes: 1

Related Questions