CODEWITHSUNDEEP
pythonmachine-learningneural-networkcomputer-visioncaffe
Ilya Ovodov
Ilya Ovodov

Reputation: 383

Caffe process label data in python layer as int

I'm trying to make label data using python layer. Namely, I have an LMDB database of caffe datums where label is composed of 6 labels using bit mask and I need to extract one of them to learn with it.

But when passed to MyPythonLayer.forward method label field appeares to have np.float32 type and is rounded so that being converted back to int has different value then original one. Naturally it makes extracting part of it with bit mask useless. Is where a way to force Caffe to pass label data to MyPythonLayer.forward as int?

MyNet.prototxt:

layer {
  name: "data"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    scale: 1
  }
  data_param {
    source: "SUMTH.../train_lmdb"
    batch_size: 3400
    backend: LMDB
  }
}
...
layer {
  name: 'convert_compose_label_py'
  type: 'Python'
  bottom: 'label'
  top: 'extracted_label'
  python_param {
    module: 'convert_compose_label'
    layer: 'ComposeLabelToSingleLabel'
    param_str: '1'
  }
}
layer {
  name: "loss"
  type: "SoftmaxWithLoss"
  bottom: "ip2"
  bottom: "extracted_label"
  top: "loss"
}

Module convert_compose_label.py:

class ComposeLabelToSingleLabel(caffe.Layer):
  ...
  def forward(self, bottom, top):
    batch_size = bottom[0].data.shape[0]
    out_file1 = open(self.log_file1, 'a')
    for i in range(batch_size):
        out_file1.write("%d\n" % bottom[0].data[i]) # result differs from label values in LMDB at 1-2 lowest digits
        single_label = (compose_label.astype(np.int32) >> self.shift) & self.kLabelMask # if not use astype(np.int32) it produces error that >> can not be used with np.float32. 
        top[0].data[i]=single_label # Obviously result is incorrect when self.shift==0
        ...

Upvotes: 1

Views: 231

Answers (1)

Shai
Shai

Reputation: 114786

No.

Why?

Caffe stores labels in Datum as int, but when it reads the Datum and "breaks" it into "data" and "label" blobs the values are converted to float32 since Blob data type is (by default) float32.

If you need multi-label input to caffe, you can use "HDF5Data" layer (that is much more flexible), or use a "Python" layer.

Upvotes: 1

Related Questions