CODEWITHSUNDEEP
pythonneural-networkconnectionsparse-matrixrecurrent-neural-network
LCheng
LCheng

Reputation: 400

How to construct a sparse recurrent neural network with limited connection in Python?

I’m trying to build a sparse recurrent neural network where there are 100 neurons in total, and each neuron is only randomly connected with 10 other neurons, and the weight is randomly drawn from a gaussian distribution with 0 mean 5e-05 standard deviation.

I know that in Python, to drawn weights from Gaussian distribution, I could use:

np.random.normal(0, 5e-05, (100, 100))

But what would be an efficient way to set up each neuron randomly connected to 10 other neurons in the network? I guess this could probably be achieved with basic python functions, without going to tensorflow or pytorch, but I’m welcoming all possible solutions.

Thanks,

Lily

Upvotes: 0

Views: 164

Answers (1)

CJR
CJR

Reputation: 3985

Use the random.choice functionality from numpy.

import pandas as pd
import numpy as np

# Create a RNG
rng = np.random.default_rng(10)

n = 100 # Number of nodes
k = 10  # Number of edges per node

# Create an empty connectivity matrix
c = np.zeros((n, n), dtype=bool)

for i in range(c.shape[0]):
    # End when all nodes have the right number of edges
    if np.all(c.sum(axis=1) == k):
        break
        
    # Select more edges from nodes with fewer edges by weighting probability
    p = 1 - c.sum(axis=1) / k
    
    # Set the probability of self-association to zero
    p[i] = 0
    
    # Choose as many edges as needed for this node to bring it up to k
    new_edges = rng.choice(np.arange(n), 
                           size=k - c[i, :].sum(), 
                           p=p / np.sum(p), 
                           replace=False)
    
    # Add the randomly selected edges for this node to a symmetric connectivity matrix
    c[i, new_edges] = True
    c[new_edges, i] = True

This gives a connectivity matrix where all the rows and columns sum to 10 edges, and the diagonal is zero (no nodes self-associate).

Upvotes: 0

Related Questions