CODEWITHSUNDEEP
deep-learningneural-networkcross-validationk-fold
zed
zed

Reputation: 11

Building neural network using k-fold cross validation

I am new to deep learning, trying to implement a neural network using 4-fold cross-validation for training, testing, and validating. The topic is to classify the vehicle using an existing dataset.

The accuracy result is 0.7.
Traning Accuracy

An example output for epochs

I also don't know whether the code is correct and what to do for increasing the accuracy.

Here is the code:

!pip install category_encoders

import tensorflow as tf
from sklearn.model_selection import KFold
import pandas as pd
import numpy as np
from tensorflow import keras
import category_encoders as ce
from category_encoders import OrdinalEncoder

car_data = pd.read_csv('car_data.csv')

car_data.columns = ['Purchasing', 'Maintenance', 'No_Doors','Capacity','BootSize','Safety','Evaluation']

# Extract the features and labels from the dataset
X = car_data.drop(['Evaluation'], axis=1)

Y = car_data['Evaluation']

encoder = ce.OrdinalEncoder(cols=['Purchasing', 'Maintenance', 'No_Doors','Capacity','BootSize','Safety'])
X = encoder.fit_transform(X)
X = X.to_numpy()

Y_df = pd.DataFrame(Y, columns=['Evaluation'])
encoder = OrdinalEncoder(cols=['Evaluation'])
Y_encoded = encoder.fit_transform(Y_df)

Y = Y_encoded.to_numpy()

input_layer = tf.keras.layers.Input(shape=(X.shape[1]))

# Define the hidden layers
hidden_layer_1 = tf.keras.layers.Dense(units=64, activation='relu', kernel_initializer='glorot_uniform')(input_layer)
hidden_layer_2 = tf.keras.layers.Dense(units=32, activation='relu', kernel_initializer='glorot_uniform')(hidden_layer_1)
# Define the output layer
output_layer = tf.keras.layers.Dense(units=1, activation='sigmoid', kernel_initializer='glorot_uniform')(hidden_layer_2)
# Create the model
model = tf.keras.Model(inputs=input_layer, outputs=output_layer)

# Initialize the 4-fold cross-validation
kfold = KFold(n_splits=4, shuffle=True, random_state=42)

# Initialize a list to store the scores
scores = []
quality_weights= []
# Compile the model
model.compile(optimizer='adam',
              loss=''sparse_categorical_crossentropy'',
              metrics=['accuracy'],
              sample_weight_mode='temporal')

for train_index, test_index in kfold.split(X,Y):
    # Split the data into train and test sets
    X_train, X_test = X[train_index], X[test_index]
    Y_train, Y_test = Y[train_index], Y[test_index]

    # Fit the model on the training data
    model.fit(X_train, Y_train, epochs=300, batch_size=64, sample_weight=quality_weights)

    # Evaluate the model on the test data
    score = model.evaluate(X_test, Y_test)

    # Append the score to the scores list
    scores.append(score[1])
    plt.plot(history.history['accuracy'])
    plt.title('Model Training Accuracy')
    plt.ylabel('Accuracy')
    plt.xlabel('Epoch')
    plt.legend(['Train'], loc='upper left')
    plt.show()
  
# Print the mean and standard deviation of the scores
print(f'Mean accuracy: {np.mean(scores):.3f} +/- {np.std(scores):.3f}')

Upvotes: 0

Views: 419

Answers (1)

Devendra Vyas
Devendra Vyas

Reputation: 95

The first thing that caught my attention was here:

model.fit(X_train, Y_train, epochs=300, batch_size=64, sample_weight=quality_weights)

Your quality_weights should be a numpy array of size of the input. Refer here: https://keras.io/api/models/model_training_apis/#fit-method

If changing that doesn't seemt to help then may be your network doesn't seem to be learning from the data. A few possible reasons could be:

  1. The network is a bit too shallow. Try adding just one more hidden layer to see if that improves anything
  2. From the code I can't see the size of your input data. Does it have enough datapoints for 4-fold cross-validation? Can you somehow augment the data?

Upvotes: 1

Related Questions