CODEWITHSUNDEEP
pythontensorflowneural-network
redacted1111
redacted1111

Reputation: 142

Value error - Balancing game input before training Neural Network

After collecting game input in the form of key combinations of w a s d and the corresponding screen image, when I try to balance the data there's a few issues. The original code had just 3 inputs of just w, a or d. I scaled this up to 9 possibilities like aw, sd or nokeys for example. Part of balancing the data is having all input vectors of the same length. But this is is where it seems to go wrong. The original code is commented out.

The balancing code:

# balance_data.py

import numpy as np
import pandas as pd
from collections import Counter
from random import shuffle
import sys

train_data = np.load('training_data-1.npy')

df = pd.DataFrame(train_data)
print(df.head())
print(Counter(df[1].apply(str)))

##lefts = []
##rights = []
##forwards = []
##
##shuffle(train_data)
##
##for data in train_data:
##    img = data[0]
##    choice = data[1]
##
##    if choice == [1,0,0]:
##        lefts.append([img,choice])
##    elif choice == [0,1,0]:
##        forwards.append([img,choice])
##    elif choice == [0,0,1]:
##        rights.append([img,choice])
##    else:
##        print('no matches')
##
##
##forwards = forwards[:len(lefts)][:len(rights)]
##lefts = lefts[:len(forwards)]
##rights = rights[:len(forwards)]
##
##final_data = forwards + lefts + rights
##shuffle(final_data)

w = []
a = []
d = []
s = []
wa = []
wd = []
sd = []
sa = []
nk = []

shuffle(train_data)

for data in train_data:
    img = data[0]
    choice = data[1]
    print(choice)

    if choice == [0,1,0,0]:
        w.append([img,choice])
    elif choice == [1,0,0,0]:
        a.append([img,choice])
    elif choice == [0,0,1,0]:
        d.append([img,choice])
    elif choice == [0,0,0,1]:
        s.append([img,choice])
    elif choice == [1,1,0,0]:
        wa.append([img,choice])
    elif choice == [0,1,1,0]:
        wd.append([img,choice])
    elif choice == [0,0,1,1]:
        sd.append([img,choice])
    elif choice == [1,0,0,1]:
        sa.append([img,choice])
    elif choice == [0,0,0,0]:
        nk.append([img,choice])
    else:
        print('no matches')

min_length = 10000
print (len(w))
print (len(a))
print (len(d))
print (len(s))
print (len(wa))
print (len(wd))
print (len(sd))
print (len(sa))
print (len(nk))

if len(w) < min_length:
    min_length = len(w)
if len(a) < min_length:
    min_length = len(a)
if len(d) < min_length:
    min_length = len(d)
if len(s) < min_length:
    min_length = len(s)
if len(wa) < min_length:
    min_length = len(wa)
if len(wd) < min_length:
    min_length = len(wd)
if len(sd) < min_length:
    min_length = len(sd)
if len(sa) < min_length:
    min_length = len(sa)

w = w[min_length]
a = a[min_length]
d = d[min_length]
s = s[min_length]
wa = wa[min_length]
wd = wd[min_length]
sd = sd[min_length]
sa = sa[min_length]
nk = nk[min_length]

final_data = w + a + d + s + wa + wd + sd + sa + nk
shuffle(final_data)

np.save('training_data-1-balanced.npy', final_data)

And the vector lengths and error after it.

9715
920
510
554
887
1069
132
128
6085
Traceback (most recent call last):
  File "C:\Users\StefBrands\Documents\GitHub\pygta5 - Copy\balance_data.py", line 115, in <module>
    sa = sa[min_length]
IndexError: list index out of range

So now mainly two things: 1. Did I make a mistake somewhere, probably yes :) 2. Is there a better way of balancing?

Upvotes: 0

Views: 31

Answers (1)

asongtoruin
asongtoruin

Reputation: 10359

You're not considering the difference between the length of a list and its maximum index - for example, the list [0, 5, 1] has length 3, but maximum index 2. As such, you should reduce the calculation of min_length by 1.

We can neaten the calculations up significantly. The lines from if if len(w) < min_length... to final_data = ... can be replaced with the following:

key_lists = (w, a, d, s, wa, wd, sd, sa, nk)
min_length = min(len(x)-1 for x in key_lists)
final_data = sum(x[min_length] for x in key_lists)

We create a tuple containing each of the lists for each key. We can then use generator expressions to find our min_length and then again to sum the values. The advantage of this is that if an additional key combo is added, we can just append its list variable to key_lists.

Upvotes: 1

Related Questions