CODEWITHSUNDEEP
opencvintelopenvino
user17535004
user17535004

Reputation:

OpenVino Performance - Neural Stick vs. CPU

I'm seeing a significant difference in inference performance between my desktop CPU and when I run on the Neural Compute Stick 2 VPU - almost 500ms slower on VPU. This is the one line that takes the most time and has the biggest difference:

result = exec_net.infer( inputs={input_layer_ir: blob} )

My desktop is my gaming machine and has a nice fast Intel CPU. That said is this the expected order of magnitude difference between the VPU and CPU?

CPU speeds are really fast like .07 seconds and VPU is around .5.

It’s the road segmentation model from the open zoo samples.

Upvotes: 0

Views: 600

Answers (1)

Rommel_Intel
Rommel_Intel

Reputation: 1413

Intel® Neural Compute Stick 2 (NCS 2) is a USB stick that offers you access to neural network functionality, without the need for large, expensive hardware. It is a plug-and-play device, so you are ready to start prototyping right away.

The performance of NCS 2 compared to the well-known CPUs or GPUs in the meaning of TFLOPS, it is still a hundred times lower. This behaviour is expected, so don’t rely on it as an external device to replace the CPU plugin.

Upvotes: 1

Related Questions