CODEWITHSUNDEEP
pythontensorflow
AlexOlsen
AlexOlsen

Reputation: 187

Tensorflow - Avoid Tensor Size Limit

I'm working on an implementation of the FCN-32 net described in the Long, Shelhamer paper, but have run into a roadblock when upsampling. In order to upsample to original size, other implementations use a conv2d_transpose layer with a bilinear filter w/kernel size 64x64. This works fine until you start using lots of classes.

For any number of classes > ~375, the filters variable in the transpose layer is > 2 gb ( 64 x 64 x (>375) x (>375) ) so Tensorflow complains and dies, saying

ValueError: Cannot create a tensor proto whose content is larger than 2GB.

Is there any way to avoid this size limit? My first thought would be generative tensor, but I can't find any documentation on how to create if such a construct exists or is possible.

Upvotes: 3

Views: 3054

Answers (1)

David Parks
David Parks

Reputation: 32071

You can split the output classes into multiple operations and concatenate them at the end.

Backprop will work just fine through the concat operation. It should be as trivial as creating two conv2d_transpose operations, each with half the classes and concat the results appropriately and continue to the loss function from there.

Creating more than 2 conv2d_transpose operations as necessary will work just as well.

After thinking about this I'm confident it will work. If there's an issue let me know and I'll update the answer.

Upvotes: 3

Related Questions