CODEWITHSUNDEEP
pythonlambdakeraskeras-layer
Axe Per
Axe Per

Reputation: 23

Keras - Modifying lambda layer after `compile()`

In Keras, how do I change a lambda layer after the model is compiled?

More specifically, let's say I want a lambda layer that computes y=a*x+b with a and b being changed every epoch. 

import keras
from keras.layers import Input, Lambda, Dense
import numpy as np


np.random.seed(seed=42)

a = 1
b = 2

def f(x, a, b):
    return a * x + b

inputs = keras.layers.Input(shape=(3,))
lam = Lambda(f, arguments={"a": a, "b": b})(inputs)
out = keras.layers.Dense(5)(lam)

model = keras.models.Model(inputs, out)
model.trainable = False
model.compile(optimizer='rmsprop', loss='mse')

x1 = np.random.random((10, 3))
x2 = np.random.random((10, 5))

model.fit(x1, x2, epochs=1)

print("Updating. But that won't work")
a = 10
b = 20
model.fit(x1, x2, epochs=1)

This returns twice loss: 5.2914 where it should return once loss: 5.2914 and then loss: 562.0562.

As far as I can tell, this seems to be an open issue, which could be remedied by writing a custom layer but I haven't been to make it work.

Any guidance is welcomed.

Upvotes: 2

Views: 823

Answers (1)

Daniel Möller
Daniel Möller

Reputation: 86620

If you work with a and b being tensors, you can change their values even after compilation.

There are two approaches. In one, you treat a and b as global vars and take them from outside the function:

import keras.backend as K

a = K.variable([1])
b = K.variable([2])

def f(x):
    return a*x + b #see the vars coming from outside here

#....

lam = Lambda(f)(inputs)

At any moment, you can manually call K.set_value(a,[newNumber]). 

K.set_value(a,[10])
K.set_value(b,[20])
model.fit(x1,x2,epochs=1)

In another approach (I don't know if there are advantages, but...it sounds at least better organized) you can make a and b be inputs to the model:

a = K.variable([1])
b = K.variable([2])
aInput = Input(tensor=a)
bInput = Input(tensor=b)

def f(x):
    return x[0]*x[1] + x[2] #here, we input all tensors in the function

#.....

lam = Lambda(f)([inputs,aInput,bInput])

You set the values of a and b the same way as the other approach.

Upvotes: 1

Related Questions