CODEWITHSUNDEEP
pythontensorflowkerasneural-networksequential
backyardigans
backyardigans

Reputation: 11

Creating a sequential model within a sequential model in TensorFlow

I have seen that it is possible to create a sequential object inside a sequential object in PyTorch. TensorFlow, from what I understand, has a different Sequential class. Can the same be done in TensorFlow? This is the code I am trying to port. Isn't this technically the same as just adding the submodel's layers to the model directly, instead of creating a sequential model inside a sequential model?

Also, this isn't the same as subclassing right?

def CORnet_Z():
    model = nn.Sequential(OrderedDict([
        ('some_layer', some_module(5, 32, kernel_size=5, stride=2)),
        ('some_other_layer', some_module(32, 32)),
        ('submodel_layer', nn.Sequential(OrderedDict([
            ('avgpool', nn.AdaptiveAvgPool2d(1)),
            ('flatten', Flatten()),
            ('linear', nn.Linear(512, 1000)),
            ('output', Identity())
        ])))
    ]))

Upvotes: 1

Views: 2025

Answers (1)

Nicolas Gervais
Nicolas Gervais

Reputation: 36624

You can pass a tf.keras.Model as a layer in tf.keras.Sequential:

import tensorflow as tf
import numpy as np

x = np.random.rand(10, 5)

model1 = tf.keras.Sequential([
    tf.keras.layers.Dense(4),
    tf.keras.layers.Dense(4)
])

model2 = tf.keras.Sequential([
    model1,
    tf.keras.layers.Dense(4)
])

model2.build(input_shape=x.shape)

model2(x)

<tf.Tensor: shape=(10, 4), dtype=float32, numpy=
array([[-0.7757652 , -0.28382677, -0.8649205 , -0.9957212 ],
       [-0.5091491 ,  0.01770999, -0.16385679, -0.1557586 ],
       [-1.1032734 , -0.03929051, -0.48151532, -0.48101106],
       [-0.10648774,  0.28984556, -0.06413348, -0.07031877],
       [ 0.32688037,  0.08868963, -0.21439247, -0.32095107],
       [ 0.06685755,  0.16916664, -0.2715247 , -0.34620026],
       [-1.2354609 , -0.03694592, -0.85326445, -0.92322665],
       [-0.09964843,  0.22206043, -0.08375332, -0.09731264],
       [-0.6423199 ,  0.23357837, -0.2716178 , -0.26587674],
       [-0.8205335 ,  0.06319132, -0.58937836, -0.64297235]],
      dtype=float32)>

No, this is not subclassing. This would be an equivalent approach by subclassing:

import tensorflow as tf
import numpy as np

x = np.random.rand(10, 5)

model1 = tf.keras.Sequential([
    tf.keras.layers.Dense(4),
    tf.keras.layers.Dense(4)
])

class SubclassedModel(tf.keras.Model):
    def __init__(self):
        super(SubclassedModel, self).__init__()
        self.base_model = model1
        self.dense_layer = tf.keras.layers.Dense(4)
        
    def call(self, inputs, training=None, mask=None):
        x = self.base_model(inputs)
        x = self.dense_layer(x)
        return x
    
sub = SubclassedModel()

sub(x)

<tf.Tensor: shape=(10, 4), dtype=float32, numpy=
array([[ 0.00611126,  0.2022147 , -0.36310855,  0.6525632 ],
       [ 0.00426999,  0.09100948, -0.23409572,  0.41532114],
       [ 0.55629945, -0.16615972, -0.77816606,  0.603715  ],
       [ 0.58559847, -0.51788765, -0.43353838, -0.08013998],
       [ 0.5025266 , -0.3441915 , -0.5166623 ,  0.11504132],
       [ 0.46883702, -0.20030999, -0.57352316,  0.50871485],
       [ 0.6971718 , -0.25111896, -0.96603405,  0.62427527],
       [-0.11085381,  0.11150448, -0.01575859,  0.39796048],
       [ 0.39540276,  0.02676181, -0.7489426 ,  0.7645229 ],
       [ 0.27323198,  0.00614326, -0.5398682 ,  0.74816173]],
      dtype=float32)>

Upvotes: 2

Related Questions