CODEWITHSUNDEEP
tensorflowtflearn
kmace
kmace

Reputation: 2044

Using a custom layer to define connectivity in tensorflow / tflearn

Instead of a fully connected layer, I would like to specify the connectivity between activation nodes using a matrix. For example:

I have a 20 node layer that is connected to a 10 node layer. Using a typical fully connected layer, my W matrix is 20 x 10, with a b vector of size 10.

My activation looks something like relu(Wx + b)

If I had a matrix of ones and zeros that was the same size as W, lets call it F, I could do a pairwise multiply between W and F to remove connections between my first layer (20 nodes) and my second layer (10 nodes)

Here is my current code:

F.shape
# (20, 10)
import tflearn
import tensorflow as tf

input = tflearn.input_data(shape=[None, num_input])

first = tflearn.fully_connected(input, 20, activation='relu')
# Here is where I want to use a custom function, that uses my F matrix
# I dont want the second layer to be fully connected to the first, 
# I want only connections that are ones (and not zeros) in F

# Currently:
second = tflearn.fully_connected(first, 10, activation='relu')
# What I want:
second = tflearn.custom_layer(first, my_fun)

Where my_fun gives me: relu( (FW)X + b) and FW is a pairwise multiplication

How do I create this function? I cant seem to find tflearn examples on how it is done, but I also know that tflearn allows for base tensorflow functions as well

Upvotes: 2

Views: 1896

Answers (1)

kmace
kmace

Reputation: 2044

Its hard to do this strictly with tflearn, but if you're willing to include base tensorflow operations, its trivial:

F.shape
# (20, 10)
import tflearn
import tensorflow as tf

input = tflearn.input_data(shape=[None, num_input])
tf_F = tf.constant(F, shape=[20, 10])

first = tflearn.fully_connected(input, 20, activation='relu')
# Here is where I want to use a custom function, that uses my F matrix
# I want only connections that are ones (and not zeros) in F

# Old:
# second = tflearn.fully_connected(first, 10, activation='relu')
# Solution:
W = tf.Variable(tf.random_uniform([20, 10]), name='Weights')
b = tf.Variable(tf.zeros([10]), name='biases')
W_filtered = tf.mul(tf_F, W)
second = tf.matmul( W_filtered, first) + b

Upvotes: 1

Related Questions