CODEWITHSUNDEEP
machine-learningpytorchonnx
maskarih
maskarih

Reputation: 867

int8 data type in Pytorch

What is the best way to run a qauntized model using int8 data types in Pytorch? I know in pytorch I can define tensors as int8, however, when I actually want to use int8, I get:

RuntimeError: _thnn_conv2d_forward is not implemented for type torch.CharTensor

So I am confused, how to run quantized model in pytorch that uses for instance int8 when the datatype is not supported for computational blocks such as convolutions? I am using pytorch version 1.0.1.post2.

Upvotes: 1

Views: 1588

Answers (2)

Qin Heyang
Qin Heyang

Reputation: 1674

In recent Pytorch versions, there is native support for quantized tensor operations. That means you can do most of common operations (Conv, ReLU, Linear, etc.) on quantized tensor (currently CPU only unfortunately). See https://pytorch.org/docs/stable/quantization-support.html

Upvotes: 0

penkovsky
penkovsky

Reputation: 993

Depends what your goals are.

  1. If you want to simulate your quantized model:

You may stick to existing float data type and only introduce truncation as needed, i.e.:

 x = torch.floor(x * 2**8) / 2**8

assuming x is a float tensor.

  1. If you want to simulate your quantized model efficiently:

Then, I am afraid PyTorch will be not very useful since the low-level convolutional operator is implemented only for float type.

Upvotes: 1

Related Questions