CODEWITHSUNDEEP
pythontensorflowmachine-learningautoencoderdimensionality-reduction
Nikhil Belure
Nikhil Belure

Reputation: 72

while training the autoencoders loss is high and constant, what am I doing wrong here

I'm trying to reduce dimension of my dataset from 154 to 20 or 30 using autoencoder.

Dataset consist of features of user behavior such as spending in last month, number of days active in last month etc. on top of it I have generate few features like cumulative spend in last 3 months, difference between spend of last month and last to last month.

I'm using the following code that I copied from kaggle notebook (https://www.kaggle.com/code/ohseokkim/dectecting-anomaly-using-autoencoder?scriptVersionId=87181418)

`

class AnomalyDetector(Model):
    def __init__(self):
        super(AnomalyDetector, self).__init__()
        self.encoder = tf.keras.Sequential([
          layers.Dense(128, activation="relu"),
          layers.Dense(64, activation="relu"),
          layers.Dense(32, activation="relu")])

        self.decoder = tf.keras.Sequential([
          layers.Dense(64, activation="relu"),
          layers.Dense(128, activation="relu"),
          layers.Dense(154, activation="sigmoid")])

    def call(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded

autoencoder = AnomalyDetector()

`

I'm getting very high loss and its not improving with epochs. What I'm doing wrong here.

Epoch 1/20
71/71 [==============================] - 1s 9ms/step - loss: 5351.0259 - val_loss: 5342.5894

I'm trying to find better ways to eliminate redundant columns without loosing important information

Upvotes: 0

Views: 168

Answers (1)

samuel fipps
samuel fipps

Reputation: 41

In my experience with auto encoders, they tend to have exploding gradients and it helps to clip them. Example with PyTorch:

optimizer.zero_grad()
loss.backward()
torch.nn.utils.clip_grad_norm_(model.parameters(), 0.5)  # this part is what I am talking about
optimizer.step()

Upvotes: 1

Related Questions