Reputation: 21
I am following the YOLO v1 paper to create an object detector from scratch with Tensorflow and python. My dataset is a set of images and a 7x7x12 Tensor that represents the label for the image. I import the image names and labels (as a string) into a dataframe from a CSV using pandas, and then do some operations on the labels to turn them into Tensors. I then create a generator using ImageGenerator.flow_from_dataframe(), and the feed that generator as the input for my model. I end up getting the following error when the model tries to call the custom loss function that I created:
File "/home/michael/Desktop/YOLO_Detector/src/main.py", line 61, in <module>
epochs=10)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 819, in fit
use_multiprocessing=use_multiprocessing)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 342, in fit
total_epochs=epochs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 128, in run_one_epoch
batch_outs = execution_function(iterator)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 98, in execution_function
distributed_function(input_fn))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 568, in __call__
result = self._call(*args, **kwds)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 615, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 497, in _initialize
*args, **kwds))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2389, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2703, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/function.py", line 2593, in _create_graph_function
capture_by_value=self._capture_by_value),
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/func_graph.py", line 978, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/eager/def_function.py", line 439, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 85, in distributed_function
per_replica_function, args=args)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 763, in experimental_run_v2
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 1819, in call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/distribute/distribute_lib.py", line 2164, in _call_for_each_replica
return fn(*args, **kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/autograph/impl/api.py", line 292, in wrapper
return func(*args, **kwargs)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 433, in train_on_batch
output_loss_metrics=model._output_loss_metrics)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 312, in train_on_batch
output_loss_metrics=output_loss_metrics))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 253, in _process_single_batch
training=training))
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_eager.py", line 167, in _model_loss
per_sample_losses = loss_fn.call(targets[i], outs[i])
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/losses.py", line 221, in call
return self.fn(y_true, y_pred, **self._fn_kwargs)
File "/home/michael/Desktop/YOLO_Detector/src/utils.py", line 25, in yolo_loss_function
pos = kb.mean(y_true-y_pred)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/math_ops.py", line 902, in binary_op_wrapper
return func(x, y, name=name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_math_ops.py", line 10104, in sub
"Sub", x=x, y=y, name=name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 576, in _apply_op_helper
param_name=input_name)
File "/home/michael/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 61, in _SatisfiesTypeConstraint
", ".join(dtypes.as_dtype(x).name for x in allowed_list)))
TypeError: Value passed to parameter 'x' has DataType string not in list of allowed values: bfloat16, float16, float32, float64, uint8, int8, uint16, int16, int32, int64, complex64, complex128
When I use the python debugger to check the y_true that is being passed to the loss function, I see the following tensor:
Tensor("IteratorGetNext:1", shape=(None, 1), dtype=string)
however when I manually check the label of the image by calling the following python code, I get a tensor with the correct shape and values:
img, label = next(train_gen)
print(type(label[0]))
print(label[0].shape)
print(label[0].dtype)
-------------Output--------------
<class 'tensorflow.python.framework.ops.EagerTensor'>
(7, 7, 12)
<dtype: 'float32'>
Below is the code that I am using to create the dataset and train the model:
import tensorflow as tf
from tensorflow import keras
import pandas as pd
import matplotlib.pyplot as plt
import cv2
import numpy as np
import os
from src import utils, model
path = "/home/michael/Desktop/YOLO_Detector/dataset/labels.csv"
train_df = pd.read_csv(path, delim_whitespace=True, header=None)
train_df.columns = ['filename', 'output_tensor']
train_df["output_tensor"] = train_df["output_tensor"].apply(lambda x: utils.string_to_tensor(x))
# train_df["output_tensor"] = train_df["output_tensor"].apply(lambda x: tf.expand_dims(x, 3))
image_generator = keras.preprocessing.image.ImageDataGenerator(rescale=1. / 255.,
validation_split=0.2)
train_gen = image_generator.flow_from_dataframe(
dataframe=train_df,
directory="/home/michael/Desktop/YOLO_Detector/dataset",
x_col='filename',
y_col='output_tensor',
class_mode='raw',
batch_size=64,
target_size=(448, 448),
shuffle=False,
subset="training"
)
validation_gen = image_generator.flow_from_dataframe(
dataframe=train_df,
directory="/home/michael/Desktop/YOLO_Detector/dataset",
x_col='filename',
y_col='output_tensor',
class_mode="raw",
batch_size=64,
target_size=(448, 448),
shuffle=False,
subset="validation"
)
img, label = next(train_gen)
print(type(label[0]))
print(label[0].shape)
print(label[0].dtype)
model = model.create_model()
lr_schedule = keras.optimizers.schedules.ExponentialDecay(
initial_learning_rate=1e-2,
decay_rate=0.0005,
decay_steps=100000
)
sgd = keras.optimizers.SGD(learning_rate=lr_schedule, momentum=0.9)
model.compile(optimizer=sgd, loss=utils.yolo_loss_function, metrics=['accuracy'])
model.fit(x=train_gen,
epochs=10)
I am using Tensorflow 2 with ROCm, and Eager Execution is on. How do I get the y_true tensor to be the correct output label (7x7x12 Tensor with dtype=float32) instead of it being a string?
Upvotes: 0
Views: 584
Reputation: 21
I figured out the issue. The issue is that you cannot correctly store a Tensor or a Numpy array in a Pandas Dataframe. I ended up having to manually create the image/tensor pairs by doing the following:
img_list = []
labels_list = []
for i in range(64):
labels_list.append(utils.string_to_numpy(train_df["output_tensor"][i]))
image = tf.keras.preprocessing.image.load_img(f"/home/michael/Desktop/YOLO_Detector/dataset/{train_df['filename'][i]}", target_size=(448, 448))
image_arr = keras.preprocessing.image.img_to_array(image) / 255.0
img_list.append(image_arr)
img = np.asarray(img_list)
label = np.asarray(labels_list)
and then calling the img as x and label as y in model.fit()
Upvotes: 0