CODEWITHSUNDEEP
pandasnumpymachine-learningtensorflowneural-network
NOOB
NOOB

Reputation: 2797

How to change dtypes of numpy array for tensorflow

I am creating a neural network in tensorflow and I have created the placeholders like this:

input_tensor = tf.placeholder(tf.float32, shape = (None,n_input), name = "input_tensor")
output_tensor = tf.placeholder(tf.float32, shape = (None,n_classes), name = "output_tensor")

During the training process, I was getting the following error:

Traceback (most recent call last):
  File "try.py", line 150, in <module>
    sess.run(optimizer, feed_dict={X: x_train[i: i + 1], Y: y_train[i: i + 1]})
TypeError: unhashable type: 'numpy.ndarray'

I identified that is because of the different datatypes of my x_train and y_train to the datatypes of the placeholders.

My x_train looks somewhat like this:

array([[array([[ 1.,  0.,  0.],
   [ 0.,  1.,  0.]])],
   [array([[ 0.,  1.,  0.],
   [ 1.,  0.,  0.]])],
   [array([[ 0.,  0.,  1.],
   [ 0.,  1.,  0.]])]], dtype=object)

It was initially a dataframe like this:

0  [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]]
1  [[0.0, 1.0, 0.0], [1.0, 0.0, 0.0]]
2  [[0.0, 0.0, 1.0], [0.0, 1.0, 0.0]]

I did x_train = train_x.values to get the numpy array

And y_train looks this:

array([[ 1.,  0.,  0.],
   [ 0.,  1.,  0.],
   [ 0.,  0.,  1.]])

x_train has dtype object and y_train has dtype float64.

What I want to know is that how I can change the datatypes of my training data so that it can work well with the tensorflow placeholders. Or please suggest if I am missing something.

Upvotes: 1

Views: 2557

Answers (2)

gngdb
gngdb

Reputation: 494

Your x_train is a nested object containing arrays, so you have to unpack it and reshape it. Here's a general purpose hack:

def unpack(a, aggregate=[]):
    for x in a:
        if type(x) is float:
            aggregate.append(x)
        else:
            unpack(x, aggregate=aggregate)
    return np.array(aggregate)
x_train = unpack(x_train.values).reshape(x_train.shape[0],-1)

Once you've got a dense array (y_train is already dense), you can use a function like the following:

def cast(placeholder, array):
    dtype = placeholder.dtype.as_numpy_dtype
    return array.astype(dtype)

x_train, y_train = cast(X,x_train), cast(Y,y_train)

Upvotes: 0

Prasad
Prasad

Reputation: 6034

It is little hard to guess what shape you want your data to be, but I am guessing one of the two combinations which you might be looking for. I will also try to simulate your data in Pandas dataframe.

df = pd.DataFrame([[[[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]]], 
[[[0.0, 1.0, 0.0], [1.0, 0.0, 0.0]]],
[[[0.0, 0.0, 1.0], [0.0, 1.0, 0.0]]]], columns = ['Mydata'])
print(df)

x = df.Mydata.values
print(x.shape)
print(x)
print(x.dtype)

Output:

                               Mydata
0  [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]]
1  [[0.0, 1.0, 0.0], [1.0, 0.0, 0.0]]
2  [[0.0, 0.0, 1.0], [0.0, 1.0, 0.0]]

(3,)
[list([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]])
 list([[0.0, 1.0, 0.0], [1.0, 0.0, 0.0]])
 list([[0.0, 0.0, 1.0], [0.0, 1.0, 0.0]])]
object

Combination 1

y = [item for sub_list in x for item in sub_list]
y = np.array(y, dtype = np.float32)
print(y.dtype, y.shape)
print(y)

Output:

float32 (6, 3)
[[ 1.  0.  0.]
 [ 0.  1.  0.]
 [ 0.  1.  0.]
 [ 1.  0.  0.]
 [ 0.  0.  1.]
 [ 0.  1.  0.]]

Combination 2

y = [sub_list for sub_list in x]
y = np.array(y, dtype = np.float32)
print(y.dtype, y.shape)
print(y)

Output:

float32 (3, 2, 3)
[[[ 1.  0.  0.]
  [ 0.  1.  0.]]

 [[ 0.  1.  0.]
  [ 1.  0.  0.]]

 [[ 0.  0.  1.]
  [ 0.  1.  0.]]]

Upvotes: 0

Related Questions