CODEWITHSUNDEEP
pythonpandasscikit-learn
Colton Allen
Colton Allen

Reputation: 3060

Training ML Model with an Array Column

I'm trying to train a model with a column that contains a serialized list of values. But I'm running into errors with the data's type. What sort of pre-processing do I need to perform before fitting the model?

TypeError: float() argument must be a string or a number, not 'list'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "main.py", line 192, in <module>
    regression = train_audio_model()
  File "main.py", line 184, in train_audio_model
    regression.fit(X_train, Y_train)
  File "/Users/colton/code/audio-analysis/env/lib/python3.6/site-packages/sklearn/linear_model/_logistic.py", line 1527, in fit
    accept_large_sparse=solver != 'liblinear')
  File "/Users/colton/code/audio-analysis/env/lib/python3.6/site-packages/sklearn/utils/validation.py", line 755, in check_X_y
    estimator=estimator)
  File "/Users/colton/code/audio-analysis/env/lib/python3.6/site-packages/sklearn/utils/validation.py", line 531, in check_array
    array = np.asarray(array, order=order, dtype=dtype)
  File "/Users/colton/code/audio-analysis/env/lib/python3.6/site-packages/numpy/core/_asarray.py", line 85, in asarray
    return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.

data.csv

Col1 | Col2
-----------
1    | 1.2,-1.3
0    | -2.5,0.9

model.py

data = pd.read_csv('data.csv', converters={'Col2': lambda x: x.split(',')})
X_train, X_test, Y_train, Y_test = train_test_split(data.drop('Col1', axis=1), data['Col1'])

regression = LogisticRegression()
regression.fit(X_train, Y_train)
return regression

data.head(2) output

                 filename                                                                                spectrogram  beep
0  ./samples/nonbeep1.wav  [-315.49462890625, 138.87547302246094, -52.60832977294922, 29.540002822875977, -2.4793...     0
1  ./samples/nonbeep2.wav  [-368.6966552734375, 167.4494171142578, -23.79843521118164, 46.0974006652832, -1.74239...     0

Upvotes: 1

Views: 1437

Answers (1)

YOLO
YOLO

Reputation: 21719

You need to split the list into separate columns. Here's a minimal example which explains this idea:

# sample df
df = pd.DataFrame({'col':[[1,2,3],[4,5,6]], 'target': [0,1]})

print(df)

         col  target
0  [1, 2, 3]       0
1  [4, 5, 6]       1

# convert column with list into separate column
df = pd.concat([df.pop('col').apply(pd.Series), df['target']], axis=1)

print(df)

   0  1  2  target
0  1  2  3       0
1  4  5  6       1

To train the model, now you can do:

X_train, X_test, Y_train, Y_test = train_test_split(df.drop('target', axis=1), df['target'])

Upvotes: 1

Related Questions