CODEWITHSUNDEEP
pythontensorflowkeras
Sec Team
Sec Team

Reputation: 57

why is the accuracy constant but loss does change?

As you can see below i have two functions , get_data() outputs a data frame for the selected asset history and passes it to train_model() every thing works fine but as the model trains the accuracy does not seem to change the loss does go down but the accuracy stays the same after the second epoch ,when training with 1000 epochs the accuracy also does not change

Things i tried changing with this code:

  1. changing unit count for each of the LSTM layers
  2. tried using differnet data frames from different sources ( alpha-vantage )
  3. changing epoch count

unfortunately nothing changed

def train_model( df):

    if not os.path.exists("/py_stuff/"):
        os.makedirs("/py_stuff/")


    checkpoint_filepath ="/py_stuff/check_point"
    weights_checkpoint = "/py_stuff/"


    checkpoint_dir = os.path.dirname(checkpoint_filepath)


    model_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(
        filepath=checkpoint_filepath,
        save_weights_only=True,
        monitor='accuracy',
        mode='max',
        save_best_only=True,
        verbose=1)


    dataset_train = df
    training_set = dataset_train.iloc[:, 1:2].values

    sc = MinMaxScaler(feature_range=(0,1))
    training_set_scaled = sc.fit_transform(training_set)

    X_train = []
    y_train = []
    for i in range(100, len(df)):
        X_train.append(training_set_scaled[i-100:i, 0])
        y_train.append(training_set_scaled[i, 0])
    X_train, y_train = np.array(X_train), np.array(y_train)
    X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))


    model = Sequential()
    model.add(LSTM(units = 100, return_sequences = True, input_shape = (X_train.shape[1], 1)))
    model.add(Dropout(0.2))

    model.add(LSTM(units=100 ,  return_sequences=True))
    model.add(Dropout(0.2))
    model.add(LSTM(units=100 , return_sequences=True))
    model.add(Dropout(0.2))
    model.add(LSTM(units=100))
    model.add(Dropout(0.2))
    model.add(Dense(units=1))
    model.compile(optimizer='adam', loss='mean_squared_error' , metrics=['accuracy'])

        ## loading weights 

    try:
        model.load_weights(checkpoint_filepath)
        print ("Weights loaded successfully $$$$$$$ ")
    except:
        print ("No Weights Found !!! ")



    model.fit(X_train,y_train,epochs=50,batch_size=100, callbacks=[model_checkpoint_callback])

    ## saving weights 


    try:
        model.save(checkpoint_filepath)
        model.save_weights(filepath=checkpoint_filepath)

        print ("Saving weights and model done ")

    except OSError as no_model:
        print ("Error saving weights and model !!!!!!!!!!!! ")





def get_data(CHOICE):
        data = yf.download(  # or pdr.get_data_yahoo(...
        # tickers list or string as well
                tickers = CHOICE,

        # use "period" instead of start/end
        # valid periods: 1d,5d,1mo,3mo,6mo,1y,2y,5y,10y,ytd,max
        # (optional, default is '1mo')
                period = "5y",

        # fetch data by interval (including intraday if period < 60 days)
        # valid intervals: 1m,2m,5m,15m,30m,60m,90m,1h,1d,5d,1wk,1mo,3mo
        # (optional, default is '1d')
                interval = "1d",

        # group by ticker (to access via data['SPY'])
        # (optional, default is 'column')
                group_by = 'ticker',

        # adjust all OHLC automatically
        # (optional, default is False)
                auto_adjust = True,

        # download pre/post regular market hours data
        # (optional, default is False)
                prepost = True,

        # use threads for mass downloading? (True/False/Integer)
        # (optional, default is True)
                threads = True,

        # proxy URL scheme use use when downloading?
        # (optional, default is None)
                proxy = None
        )

        dff = pd.DataFrame(data)
        return dff





df = get_data(CHOICE="BTC-USD")



train_model(df)

Upvotes: 0

Views: 220

Answers (2)

Philippe Remy
Philippe Remy

Reputation: 3123

You are dealing with a regression problem where the accuracy is not defined.

The accuracy is defined as the probability of belonging to a specific class. For example, the probability of the output to be the digit 9. The number of classes is finite (or countable).

In your case, your network outputs a real number. The notion of accuracy does not make any sense in this context.

For example, the probability of your output to be 1.000 for example is 0. Although (and surprisingly!), a probability of zero does not mean that the event will never happen!

Ideally, Keras should return an error saying accuracy not defined.

Upvotes: 1

Nima Aghli
Nima Aghli

Reputation: 484

From your loss function, it looks like you have a regression network. Your loss is Mean Squared Error and the metric accuracy does not have any meaning for regression networks. Accuracy metric is only meaningful when used for classification models. So you can remove the metrics=['accuracy'] from your compile code and and use loss value to evaluate your model. So if the loss is decreasing that means your optimizer is successfully training the network.

Upvotes: 1

Related Questions