CODEWITHSUNDEEP
numpytensorflowrandommachine-learningdataset
Sim1bet
Sim1bet

Reputation: 33

Randomization of patterns in a dataset

I am pretty new to the ML world and I am trying to randomize the datasets X_Train, Y_Train to later build minibatches from them. The randomization of X_Train works fine, but that of Y_Train keeps giving me traceback:

can only concatenate list (not "int") to list

import numpy as np
import math

def create_datasets():
   dataset = np.genfromtxt('winequality-white.csv', delimiter=';')
   dataset = dataset[1:,:]
   X_Test=dataset[:64,:-1]
   X_Train=dataset[64:,:-1]
   m = X_Train.shape[0]
   Y_Test=dataset[:64,-1:]
   Y_Train=dataset[64:,-1:].reshape(m,1)

   return X_Train, Y_Train, X_Test, Y_Test, m

def shuffling(X_Train,Y_Train,m,minibatch_size):
   permutation=list(np.random.permutation(m))
   shuffled_X=X_Train[permutation,:].T
   shuffled_Y=Y_Train[permutation,:]
   n_comp_minibatches=math.floor(m/minibatch_size)    #Total n. of minibatches with 64 elements
   minibatches=[]

Is anybody able to tell me where I am wrong?

Upvotes: 2

Views: 68

Answers (1)

Maxim
Maxim

Reputation: 53758

Your code seems fine to me. Here's a complete version:

def shuffling(X_Train, Y_Train, m, minibatch_size):
    permutation = list(np.random.permutation(m))
    shuffled_X = X_Train[permutation, :]
    shuffled_Y = Y_Train[permutation, :]
    n_comp_minibatches = math.floor(m / minibatch_size)
    minibatches = [(shuffled_X[i*minibatch_size:(i+1)*minibatch_size],
                    shuffled_Y[i*minibatch_size:(i+1)*minibatch_size])
                   for i in range(n_comp_minibatches)]
    return minibatches

If you want to include the last incomplete mini-batch as well, simply use n_comp_minibatches + 1 in the list comprehension. Also it's usually more convenient to work with [rows, features] data rather than [features, data], that's why I skipped the transposition, but it's up to you.

Upvotes: 1

Related Questions