CODEWITHSUNDEEP
pythontensorflowneural-network
lior1zh2000
lior1zh2000

Reputation: 113

module 'tensorflow' has no attribute 'get_collection'

I want to add regularization to my tf neural network:

I have tried the first solution (Lukazs's solution) of:

How to add regularizations in TensorFlow?

But then the compiler yield at me:

module 'tensorflow' has no attribute 'get_collection'?

How I need add this to the module? Is there another way to add regularization?

This is my relevant part of code:

def get_trained_model(X,y,hidden_size_list, steps, lambdaa = 0):
    model = keras.models.Sequential()
    model.add(keras.layers.Flatten(input_shape = (X.shape[1],)))
    for hs in hidden_size_list:
        model.add(keras.layers.Dense(hs, activation = 'relu'))
    model.add(keras.layers.Dense(2))
    
    my_normal_loss = keras.losses.SparseCategoricalCrossentropy(from_logits = True)
    reg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
    reg_constant = lambdaa  # Choose an appropriate one.
    loss = my_normal_loss + reg_constant * sum(reg_losses)

    optim = keras.optimizers.Adam(learning_rate = 0.001) #lrening_rate
    metrics = ["accuracy"]
    model.compile(loss = loss, optimizer = optim, metrics = metrics)
    batch_size = X.shape[0]
    model.fit(X, y, batch_size = batch_size, epochs = steps, shuffle = True, verbose =1)

    return mode

This is the error:

AttributeError                            Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 nuearal_network_1(train_data, test_data, [20,20,20,20,20], 0)

File ~\machine_learning\kaggle_competitions\spaceship-titanic\final_code\submissions_creation.py:125, in nuearal_network_1(train_data, test_data, hidden_size_list, lambdaa, save_name, submission_ex)
    122 save_name += "/lambdaa_" + str(lambdaa)
    124 steps = 3000
--> 125 model = get_trained_model(X,y,hidden_size_list, steps, lambdaa)
    126 predictions = get_prediction(x_test, model)
    128 data = pd.read_csv(submission_ex)

File ~\machine_learning\kaggle_competitions\spaceship-titanic\final_code\neural_network.py:15, in get_trained_model(X, y, hidden_size_list, steps, lambdaa)
     12 model.add(keras.layers.Dense(2))
     14 my_normal_loss = keras.losses.SparseCategoricalCrossentropy(from_logits = True)
---> 15 reg_losses = tf.get_collection(tf.GraphKeys.REGULARIZATION_LOSSES)
     16 reg_constant = lambdaa  # Choose an appropriate one.
     17 loss = my_normal_loss + reg_constant * sum(reg_losses)

AttributeError: module 'tensorflow' has no attribute 'get_collection'

Upvotes: 0

Views: 1286

Answers (1)

Ninad Kulkarni
Ninad Kulkarni

Reputation: 68

from tensorflow.keras import layers
from tensorflow.keras import regularizers

layer = layers.Dense(
    units=64,
    kernel_regularizer=regularizers.L1L2(l1=1e-5, l2=1e-4),
    bias_regularizer=regularizers.L2(1e-4),
    activity_regularizer=regularizers.L2(1e-5)
)

https://keras.io/api/layers/regularizers/

You can refer to this link for adding the regularization for the layers. Keras has the inbuilt regularization function for use.

Upvotes: 2

Related Questions