CODEWITHSUNDEEP
machine-learningcomputer-visionneural-networkdeep-learningcaffe
experimenter
experimenter

Reputation: 63

InfogainLoss layer blobs failure

I am trying to build up caffe's infogain loss layer to work. I have seen the posts, solutions but for me it's still doesn't work

My data lmdb dimensions are Nx1xHxW (grayscale images) and my target image lmdb dimensions are Nx3xH/8xW/8 (rgb images). My last convolutional layer's dimension is 1x3x20x80. The output_size is 3, so I have 3 classes as my label numbers are (0,1,2) in the target lmdb image dataset.

I want to try out infogain loss layer, because I think I have class imbalance problem. Most of my images contains too much background.

After My last convolutional layer (conv3) I have these:

 layer {
    name: "loss"
    type: "SoftmaxWithLoss"
    bottom: "conv3"
    top: "loss"
 } 


 layer {
    bottom: "loss"
    bottom: "label"
    top: "infoGainLoss"
    name: "infoGainLoss"
    type: "InfogainLoss"
    infogain_loss_param {
      source: "infogainH.binaryproto"
    }
 }

My infogain matrix was generated by InfogainLoss layer post (as Shai suggested) so my H matrix is 1x1x3x3 dimension (an identity matrix). So my L is 3 as I have 3 classes. When I run the prototxt file everything is fine (dimensions are ok), but after my last convolution layer (conv3 layer) I get the following error:

 I0320 14:42:16.722874  5591 net.cpp:157] Top shape: 1 3 20 80 (4800)
 I0320 14:42:16.722882  5591 net.cpp:165] Memory required for data:   2892800
 I0320 14:42:16.722892  5591 layer_factory.hpp:77] Creating layer loss
 I0320 14:42:16.722900  5591 net.cpp:106] Creating Layer loss
 I0320 14:42:16.722906  5591 net.cpp:454] loss <- conv3
 I0320 14:42:16.722913  5591 net.cpp:411] loss -> loss
 F0320 14:42:16.722928  5591 layer.hpp:374] Check failed: ExactNumBottomBlobs() == bottom.size() (2 vs. 1) SoftmaxWithLoss Layer takes 2 bottom blob(s) as input.

I double checked, every lmdb dataset filename has been set correctly. I don't know what can be the problem. Any idea?

Dear @Shai

Thank you for your answer. I did the following as you mentioned:

 layer {
    name: "prob"
    type: "Softmax"
    bottom: "conv3"
    top: "prob"
    softmax_param { axis: 1 }
 }

 layer {
    bottom: "prob"
    bottom: "label"
    top: "infoGainLoss"
    name: "infoGainLoss"
    type: "InfogainLoss"
    infogain_loss_param {
        source: "infogainH.binaryproto"
    }
 }

But I still have error:

Top shape: 1 3 20 80 (4800)
I0320 16:30:25.110862  6689 net.cpp:165] Memory required for data: 2912000
I0320 16:30:25.110867  6689 layer_factory.hpp:77] Creating layer infoGainLoss
I0320 16:30:25.110877  6689 net.cpp:106] Creating Layer infoGainLoss
I0320 16:30:25.110884  6689 net.cpp:454] infoGainLoss <- prob
I0320 16:30:25.110889  6689 net.cpp:454] infoGainLoss <- label
I0320 16:30:25.110896  6689 net.cpp:411] infoGainLoss -> infoGainLoss
F0320 16:30:25.110965  6689 infogain_loss_layer.cpp:35] Check failed: bottom[1]->height() == 1 (20 vs. 1)

Upvotes: 3

Views: 1742

Answers (1)

Shai
Shai

Reputation: 114866

What went wrong?

Your error comes from "loss" layer, not the "InfogainLoss" layer: You confused "Softmax" layer that outputs class probabilities, with "SoftmaxWithLoss" layer that outputs a (scalar) loss value.

What should you do?

  1. Replace the "loss" layer with a "prob" layer of type "Softmax" layer:

    layer {
  name: "prob"
  type: "Softmax" # NOT SoftmaxWithLoss
  bottom: "conv3"
  top: "prob"
  softmax_param { axis: 1 } # compute prob along 2nd axis
}

  2. You need to compute the loss across the second dimension, currently it seems like "InfogainLoss" layer does not support this feature. You might need to tweak "InfogainLoss" layer to have functionality like "SoftmaxWithLoss" that allows loss to be computed along arbitrary axis.
    Update: I created a pull request on BVLC/caffe that "upgrades" infogain loss layer. This upgraded version supports "loss along axis" like you are after. Moreover, it makes the "Softmax" layer redundant as it computes the probabilities internally (see this thread).
    The upgraded layer can be used like this:

    layer {
  bottom: "conv3" # prob is computed internally
  bottom: "label"
  top: "infoGainLoss"
  name: "infoGainLoss"
  type: "InfogainLoss"
  infogain_loss_param {
    source: "infogainH.binaryproto"
    axis: 1  # compute loss and probability along axis
  }
}

Upvotes: 4

Related Questions