Load (pretrained) CNTK model using C#

Question

I am using CNTK.GPU v2.2.0 and saved a model using the following method:

model.Save(modelFilePath);

Now I want to load it again and e.g. continue training or just evaluate samples. I can see two ways how this could be possible. One way works, but is impracticable. The second one does not work.

1. I build the whole structure of my neural network from scratch again and then I call the following method on it:

   model.Restore(modelFilePath);
   

Indeed, this works.

2. I create my model using the following static method:

   Function.Load(modelFilePath, DeviceDescriptor.GPUDevice(0));
   

This does not work.

After these actions I just create a trainer for the model, create a minibatchSource and try to train the model the same way I did before I saved the model.

But with the second strategy I get the following exception:

System.ArgumentOutOfRangeException: 'Values for 1 required arguments 'Input('features', [28 x 28 x 1], [, #])', that the requested output(s) 'Output('aggregateLoss', [], []), Output('lossFunction', [1], [, #]), Output('aggregateEvalMetric', [], [])' depend on, have not been provided.

[CALL STACK]
    > CNTK::Internal::  UseSparseGradientAggregationInDataParallelSGD
    - CNTK::Function::  Forward
    - CNTK::  CreateTrainer
    - CNTK::Trainer::  TotalNumberOfSamplesSeen
    - CNTK::Trainer::  TrainMinibatch (x2)
    - CSharp_CNTK_Trainer_TrainMinibatch__SWIG_0
    - 00007FFA34AE8967 (SymFromAddr() error: The specified module could not be found.)

It says that the input features have not been provided. I am using the input when training and when creating the model by scratch:

var input = CNTKLib.InputVariable(_imageDimension, DataType.Float, _featureName);
var scaledInput = CNTKLib.ElementTimes(Constant.Scalar<float>(0.002953125f, _device), input);
...

So I thought I would have to replace the input of the loaded model with the one I create for training, and use when I create the model by scratch - although the input is not different. But I stuck at trying this because I could not retrieve the input of the model object, which I would need for replacement (I think).

model.FindByName(inputLayerName);

just returns null, although I clearly can see that the name matches with the layer name in the model's "Inputs" List in the debugger.

In consequence I do not know how to properly load a saved model. I hope someone can help me.

Answer 1

Luckily I just found the answer by myself. I'll post it here, because there are probably other CNTK beginners, who might stumble about that issue or generally want to know how you load a model properly.

The problem was that I did not use the same input object for training and the model creation. In other words, if I let create my model by the mentioned static method, I still have to ensure that the object in the model and the one used for training are the same. This should be possible by the following ways:

1. Replace the input of the loaded model with your own input object and use this one also for training. I did not test that, but it should work.

2. Extract the input of the loaded model and use that one for training. I just tested this and it works. There is the code I use:

   var labels =
       CNTKLib.InputVariable(new int[] {_classesNumber}, DataType.Float, _labelNa
   
   Variable input;
   Function model;
   if (File.Exists(_modelFile))
   {
       model = Function.Load(_modelFile, DeviceDescriptor.GPUDevice(0));
       input = model.Arguments.Single(a => a.Name == _featureName);
   }
   else
   {
       input = CNTKLib.InputVariable(_imageDimension, DataType.Float, _featureName);
       model = BuildNetwork(input);
   }
   
   var trainer = CreateTrainer(model, labels);
   
   IList<StreamConfiguration> streamConfigurations = new StreamConfiguration[]
   {
       new StreamConfiguration(_featureName, _imageSize), 
       new StreamConfiguration(_labelName, _classesNumber)
   };
   
   var minibatchSource = MinibatchSource.TextFormatMinibatchSource(
       Path.Combine(_ressourceFolder, _trainingDataFile),
       streamConfigurations,
       MinibatchSource.InfinitelyRepeat);
   
   TrainModel(minibatchSource, trainer, labels, input);
   

One mistake I made in the beginning, too, was to use

Variable layer model.FindByName(inputLayerName)

although I had to use

Variable layer = model.Arguments.Single(a => a.Name == inputLayerName);