CODEWITHSUNDEEP
pythonnumpytensorflowkerasregression
asendjasni
asendjasni

Reputation: 1084

keras' preprocess_input led to ValueError: Data cardinality is ambiguous

I'm using keras for a regression model. I'm trying to fine tune the VGG16. Everything was working. After I added the tensorflow.keras.applications.vgg16.preprocess_input to preprocess the images according to the VGG16 I got the following error :

ValueError: Data cardinality is ambiguous: x sizes: 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 y sizes: 422 Make sure all arrays contain the same number of samples.

After loading the images, I loop through theme and apply this:

from keras.preprocessing.image import img_to_array

images = []
for img in loaded_images:
    img = img_to_array(img)
    img = img.reshape((1, img.shape[0], img.shape[1], img.shape[2]))
    img = preprocess_input(img)
    images.append(img)

actual = actual.reshape(1,-1)
assert len(images) == len(actual)

After that I partition the images into train, valid and testing set as follows:

Edit:

train_images,test_images, train_actual_score , test_actual_score = train_test_split(images, scaled_train_mos, test_size=0.20, random_state=42)

Finally I call the fit func :

history = model.fit(x=train_images, y=train_actual_score, validation_data=(test_images, test_actual_score), epochs=30, batch_size=8, verbose=2)

Update:

Here is a standalone code to reproduce the above issue.

Upvotes: 0

Views: 442

Answers (1)

Aniket Bote
Aniket Bote

Reputation: 3574

There are few changes that I did to your code in order to make it work:

1st: In generating your random data you were generating img of size (244,24,3) however your model takes input as (224,224,3)

# original
im = np.random.randint(0,255,(244,244,3)) 
# changes
im = np.random.randint(0,255,(224,224,3))

2nd: Changed reshape function:

# Original
from keras.preprocessing.image import img_to_array
images_ = []
for img in images:
    img = img_to_array(img)
    img = img.reshape(1, img.shape[0], img.shape[1], img.shape[2])
    img = preprocess_input(img)
    images_.append(img)
# changed
from keras.preprocessing.image import img_to_array

images_ = []
for img in images:
    img = img_to_array(img)
    img = img.reshape(img.shape[0], img.shape[1], img.shape[2])
    img = preprocess_input(img)
    images_.append(img)

The reshape method you used was generating (50, 1, 244, 244, 3) which is not correct. Although I'm not sure why you used the reshape in the first place since your data is already in the correct format.

3rd: You reshape the scores into shape (1,50) which means your training data contains only 1 sample and 50 values. There is no need to change the shape of scores. (50,1) is the correct shape representing you have 50 samples and prediction value.

4th: Converted the list to numpy array:

train_images = np.array(train_images)
test_images = np.array(test_images)
train_score = np.array(train_score)
test_score = np.array(test_score)

See the full working code here

Upvotes: 1

Related Questions