CODEWITHSUNDEEP
tensorflow
Kisme
Kisme

Reputation: 41

Is tf.python_io.TFRecordWriter thread-safe?

I want to convert csv format dataset files to TFRecords parallelly. Is tf.python_io.TFRecordWriter thread-safe?

Upvotes: 4

Views: 608

Answers (1)

Kousuke Kuzuoka
Kousuke Kuzuoka

Reputation: 125

I was also searching for the question. I found at it is not a thread safe method. I couldn't find any documentation for this answer but tried both multi threading process and single threading process instead on same data. The result was that multi threading approach creates a record file that throws an error while training and single threading approach creates a valid record file which of course runs fine while training but takes much longer time than multi threading during creation time. I hope this answer is helpful.

Upvotes: 1

Related Questions