CODEWITHSUNDEEP
pythontensorflowkeras
Tony Ennis
Tony Ennis

Reputation: 12299

expand a tensor in Keras

My hare-brained idea is to create a custom layer that allows me to programmatically add features to a model's output.

EDIT - My "O" output values (see image below) are ASCII values. I want the "F"eature nodes to be 1 if the corresponding "O" nodes are alphabetic and 0 otherwise. In a previous experiment, the additional information made training much much better.

class Unpack_and_Categorize(keras.layers.Layer):
    
    def __init__(self, units=32, **kwargs):
        super(Unpack_and_Categorize, self).__init__(units, **kwargs)
        self.units = units
        self.trainable = False

    def build(self, input_shape):
        self.weight = self.add_weight(
            shape=(input_shape[-1], self.units),
            trainable=True,
        )
        self.bias = self.add_weight(
            shape=(self.units,), trainable=True, dtype="float32"
        )

    def call(self, inputs):
        batch_size = inputs.shape[0]
        one_hot_size = self.units
        c = tf.constant([0] * (one_hot_size * batch_size), shape=(batch_size, one_hot_size))
        base_out = tf.tensordot(inputs, self.weight, axes = 1) + self.bias
        return tf.concat(base_out, c, shape=(batch_size, 2*one_hot_size))

This image shows what I am trying to accomplish. My custom layer (right side) has 3 values that are densely connected to the previous layer. But now I want to add three 3 more output values that are totally derived from the O1..3. For example, I might set Fx to 1 if Ox was an even number. This would be done in the call method.

So the challenge is that I don't want to hardcode the number of outputs. That is, if the input layer has 10 inputs, then the customer layer will have 20 values. (The challenge that follows is 'will it back-prop, or simply explode...)

enter image description here

Here is an example where we see the "A" is categorized with a 1, while the punctuation and numeric are categorized with a 0.

enter image description here

Upvotes: 1

Views: 182

Answers (1)

AloneTogether
AloneTogether

Reputation: 26708

It's a bit difficult to say exactly what you want to do without seeing all your code, but you need to make sure that all dimensions except the one you want to concatenate are the same:

import tensorflow as tf

def call(inputs):
    w_init = tf.random_normal_initializer()
    w = tf.Variable(
            initial_value=w_init(shape=(10, 10), dtype="float32"),
            trainable=True,
        )
    batch_size = tf.shape(inputs)[0]
    c = tf.constant([1.0] * (15 * tf.cast(batch_size, tf.float32)), shape=(batch_size, 15))
    print('Inputs: ',  inputs.shape)
    print('C: ', c.shape)
    return tf.concat((tf.tensordot(inputs, w, axes = 1) ,c), 1)

batch_size = 5
print('Result: ', call(tf.random.normal((batch_size, 10))).shape)

Inputs:  (5, 10)
C:  (5, 15)
Result:  (5, 25)

Upvotes: 1

Related Questions