CODEWITHSUNDEEP
machine-learningdeep-learningcaffeconv-neural-network
mcExchange
mcExchange

Reputation: 6498

How to get features from several layers using c++ in caffe

How can I get both the 4096 dim feature layer and the 1000 dim class layer in caffe after one forward pass using C++?

I tried to look it up in extract_features.cpp but it uses some weird datum object, so I cannot really understand how it works.

So far I was simply cropping my prototxt files up to the layer that I wanted to extract and used

[...]
net->ForwardPrefilled();
Blob<float> *output_layer = net->output_blobs()[0];
const float *begin = output_layer->cpu_data();
const float *end = begin + output_layer->channels();
return vector<float>(begin, end);

but that does not work if I want to extract two specific layers (eg "prob" and "fc7") simultaneously.

Upvotes: 2

Views: 1558

Answers (1)

Dale
Dale

Reputation: 1608

Update

The simple work flow of extract_feature.cpp(suppose you have a shared_ptr<Net<float> > net object in c++):

  1. perform net forward to process input: net->Forward(). In this step, there is a Data layer in the net to read the input images. So if in your own app/code you want read an image to cv::Mat image and feed it into net, you can write a code like:

    // for data preprocess
shared_ptr<caffe::DataTransformer<float> > data_transformer;
caffe::TransformationParameter trans_para;
// set mean
trans_para.set_mean_file("/path/to/image_mean.binaryproto");
// set crop size, e.g.here is cropping 227x227
trans_para.set_crop_size(227);
// instantiate a DataTransformer using trans_para for image preprocess
data_transformer.reset(new caffe::DataTransformer<float>(trans_para, caffe::TEST));
const std::vector<caffe::Blob<float> *> net_input = net->input_blobs();
// maybe you need to resize image before this step
data_transformer->Transform(image, *net_input[0]);
net->Forward();

    And the net.prototxt should have a Input layer as the first layer, e.g. this deploy.prototxt.

  2. get the feature blobs according to their names:const boost::shared_ptr<Blob<Dtype> > feature_blob = net->blob_by_name(blob_names[i])

  3. extract the feature data from the blob you get into a structure you want, e.g. an arry, a simple sample code can be:

    count = feature_blob->channels() * feature_blob->height() * 
    feature_blob->width();
float* feature_array = new float[count]; 
const float* feature_blob_data = feature_blob->cpu_data() +
    feature_blob->offset(n); // feature data generated from 
                             // the nth input image within a batch 
memcpy(feature_array, feature_blob_data, count * sizeof(float)); 
...// other operations
delete [] feature_array;

Note that the data stored from feature_blob_data is in row-major order.

The extract_feature.cpp's usage should be like this for your task:

path/to/extract_features your_pretrained_model.caffemodel \
    net.prototxt 4096_dim_feature_blob_name,1000_dim_class_feature_blob_name \
    saved_4096_dim_feature_database,saved_1000_dim_class_feature_database \
    num_mini_batches(times for forward pass) lmdb(or leveldb) GPU(or CPU)

The net.prototxt should contain a data layer that can read the input image data.

And when running, it will first the read image data from the data layer within net.prototxt and perform num_mini_batches times of forward pass and extract the 2 two feature blob 4096_dim_feature_blob_name, 1000_dim_class_feature_blob_name's data into a structure typed of Datum and then serialize them to save in the database saved_4096_dim_feature_database, saved_1000_dim_class_feature_database which are typed of lmdb or leveldb.

When finished, you can read the saved feature data from saved_4096_dim_feature_database, saved_1000_dim_class_feature_database using a data layer in net.prototxt respectively.

BTW, datum is a structure that can store at most 4D data as well as the data's shape and label information etc. It is defined in caffe.proto, generated using google protobuf and is convenient for data interchange between caffe and database like LMDB and LEVELDB.

Upvotes: 2

Related Questions