TensorFlow Model not performing any training

Question

I am training a simple machine learning model that takes a 1D description of a physical system (502 elements) and predicts the total energy (1 element). As I am new to TensorFlow I have used a simple dense neural network with two hidden layers of 64 neurons each:

Model: "total_energy"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
charge_density_x_max (InputL [(None, 502)]             0         
_________________________________________________________________
hidden_1 (Dense)             (None, 64)                32192     
_________________________________________________________________
hidden_2 (Dense)             (None, 64)                4160      
_________________________________________________________________
dense (Dense)                (None, 1)                 65        
=================================================================
Total params: 36,417
Trainable params: 36,417
Non-trainable params: 0
_________________________________________________________________

This is my source code for the training, evaluation and prediction:

# imports
import os
import ast
import numpy as np
import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt

# load the dataset from the csv file
data = pd.read_csv('1e_data.csv')

# load in the data
x_train = np.zeros(shape=(600, 502))
x_test = np.zeros(shape=(400, 502))
y_train = np.zeros(shape=(600))
y_test = np.zeros(shape=(400))
for i in range(0, 1000):
    if i < 600:
        x_train[i,:] = np.append(np.array(ast.literal_eval(data.loc[i,'n'])), float(data.loc[i,'xmax']))
        y_train[i] = float(data.loc[i,'E'])
    else:
        x_test[i-600,:] = np.append(np.array(ast.literal_eval(data.loc[i,'n'])), float(data.loc[i,'xmax']))
        y_test[i-600] = float(data.loc[i,'E'])

# build the neural network model
inputs = tf.keras.Input(shape=(502,), name='charge_density_x_max')
hidden1 = tf.keras.layers.Dense(64, activation='sigmoid', name='hidden_1')(inputs)
hidden2 = tf.keras.layers.Dense(64, activation='sigmoid', name='hidden_2')(hidden1)
outputs = tf.keras.layers.Dense(1)(hidden2)
model = tf.keras.Model(inputs=inputs, outputs=outputs, name='total_energy')

# save the info of the model
with open('model_info.dat','w') as fh:
    model.summary(print_fn=lambda x: fh.write(x + '\n'))

# compile the model
model.compile(optimizer='adam', loss='mean_absolute_percentage_error', metrics=['accuracy'])

# perform the training
model.fit(x_train, y_train, epochs=10)

# evaluate the model for accuracy
model.evaluate(x_test, y_test, verbose=2)

Yet when I run this it seems to do no training at all, giving an accuracy of 0.0000e+00:

Epoch 1/10                                                                                     
600/600 [==============================] - 0s 196us/sample - loss: 289.0616 - acc: 0.0000e+00  
Epoch 2/10                                                                                     
600/600 [==============================] - 0s 37us/sample - loss: 144.5967 - acc: 0.0000e+00   
Epoch 3/10                                                                                     
600/600 [==============================] - 0s 46us/sample - loss: 97.2109 - acc: 0.0000e+00    
Epoch 4/10                                                                                     
600/600 [==============================] - 0s 46us/sample - loss: 108.0698 - acc: 0.0000e+00   
Epoch 5/10                                                                                     
600/600 [==============================] - 0s 47us/sample - loss: 84.5921 - acc: 0.0000e+00    
Epoch 6/10                                                                                     
600/600 [==============================] - 0s 38us/sample - loss: 79.9309 - acc: 0.0000e+00    
Epoch 7/10                                                                                     
600/600 [==============================] - 0s 38us/sample - loss: 80.6755 - acc: 0.0000e+00    
Epoch 8/10                                                                                     
600/600 [==============================] - 0s 47us/sample - loss: 87.5954 - acc: 0.0000e+00    
Epoch 9/10                                                                                     
600/600 [==============================] - 0s 46us/sample - loss: 73.6634 - acc: 0.0000e+00    
Epoch 10/10                                                                                    
600/600 [==============================] - 0s 38us/sample - loss: 78.0825 - acc: 0.0000e+00    
400/400 - 0s - loss: 70.3813 - acc: 0.0000e+00                                                 

I have probably made a simple mistake here, but I do not know how to begin debugging. This should perform at least some training, but at the moment it seems to just skip the training and give an accuracy of 0.

Answer 1

You are in a regression setting, where accuracy is meaningless (it is meaningful only for classification problems); see What function defines accuracy in Keras when the loss is mean squared error (MSE)? for more details (it is applicable in your case, too, despite the use of a different loss).

The fact that your network does indeed learn is apparent from the reduction in your loss, which is the actual quantity of interest in regression problems (you simply don't need any metrics here).

Independently of the above, you should probably change the sigmoid activations to relu (we normally do not use sigmoid nowadays for intermediate layers).