CODEWITHSUNDEEP
pythoncomputer-visionintelopenvinoyolov5
Alexandre Tavares
Alexandre Tavares

Reputation: 113

Yolov5s on Openvino

I have trained a model using yolov5 and it is working just fine:

Model's result - Yolov5 script - pytorch

My ultimate goal is to use a model that I have trained on custom data (to detect the hook and bucket) in the Openvino framework.

To achieve this, I first exported the best version of the model to the appropriate Openvino format, using the following command:

!python export.py --weights runs/train/yolov5s24/weights/best.pt --include openvino  --dynamic --simplify

The export performed successfully generated me 3 files: best.xml, best.bin, best.mapping;

Now I would like to load it using the Openvino framework and to to that I am following this pipeline:

  1. Create Core object
  2. 1.1. (Optional) Load extensions
  3. Read a model from a drive
  4. 2.1. (Optional) Perform model preprocessing
  5. Load the model to the device
  6. Create an inference request
  7. Fill input tensors with data
  8. Start inference
  9. Process the inference results

1 Create Core

import numpy as np
import openvino.inference_engine as ie
core = ie.IECore()

2 Read a model from a drive

path_to_xml_file = 'models/best_openvino_model/best.xml'
path_to_bin_file = 'models/best_openvino_model/best.bin'
network = core.read_network(model=path_to_xml_file, weights=path_to_bin_file)

3 Load the Model to the Device

# Load network to the device and create infer requests
exec_network = core.load_network(network, "CPU", num_requests=4)

And here I am getting an error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-21-7c9ba5f53484> in <module>
      1 # Load network to the device and create infer requests
----> 2 exec_network = core.load_network(network, "CPU", num_requests=4)

ie_api.pyx in openvino.inference_engine.ie_api.IECore.load_network()

ie_api.pyx in openvino.inference_engine.ie_api.IECore.load_network()

RuntimeError: Check 'std::get<0>(valid)' failed at inference/src/ie_core.cpp:1414:
InferenceEngine::Core::LoadNetwork doesn't support inputs having dynamic shapes. Use ov::Core::compile_model API instead. Dynamic inputs are :{ input:'images,images', shape={?,3,?,?}}

I am using the OpenVINO™ Development Tools - release 2022.1; The files to reproduce the error are here;

Upvotes: 1

Views: 1519

Answers (2)

markusbr
markusbr

Reputation: 118

You use API 1.0 that does not support dynamic shapes.

You need to use API 2.0 for dynamic shapes.

Here an example of how to infer for dynamic shapes using API 2.0:

from openvino.runtime import AsyncInferQueue, Core, InferRequest, Layout, Type
core = Core()
model = core.read_model("model.xml")
compiled_model = core.compile_model(model, "CPU")
output = compiled_model.infer_new_request({0: input_data})

Upvotes: 1

Zul_Intel
Zul_Intel

Reputation: 66

This error is expected as the model contains a dynamic shape. This model can be executed using the ov::Core:compile_model API in OpenVINO 2022.1.

You can refer to class ov::CompiledModel and Dynamic Shape for more information.

Upvotes: 2

Related Questions