CODEWITHSUNDEEP
pythontensorflow
Pratik Kumar
Pratik Kumar

Reputation: 2231

Prediction with tensorflow-gpu is slower than tensorflow-cpu

Below is my code for prediction:

start=time.time()
with tf.Session(graph=graph) as sess:
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)
    predict('/home/4_bikes/test_images/bikerider4.jpg',sess)    
stop=time.time()
print('Time taken for prediction :: {}'.format(stop-start))

Below is my predict function :

def predict(file_name,sess):

  t = read_tensor_from_image_file(
      file_name,
      input_height=input_height,
      input_width=input_width,
      input_mean=input_mean,
      input_std=input_std)

  results = sess.run(output_operation.outputs[0], {
        input_operation.outputs[0]: t
    })
  results = np.squeeze(results)

  index=results.argmax()

  prediction=labels[index]
  bike_predictor = bike_classifier()
  if prediction == 'bikes':
    bike_predictor.predict(t)
  else:
    print('Predicted as :: unknown')

I have installed tensorflow-gpu on python-2 and tensorflow-cpu on python-3. When I run it with tensorflow-gpu I get:

Time taken for prediction :: 2.92091107368

When I run with tensorflow-cpu I get:

Time taken for prediction :: 1.7942276000976562

I am sure I am using GPU because when running with python-2 I get log:

name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.6705
pciBusID: 0000:01:00.0
totalMemory: 10.91GiB freeMemory: 10.28GiB
2018-05-31 18:23:26.762628: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-05-31 18:23:26.906629: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-05-31 18:23:26.906672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 
2018-05-31 18:23:26.906679: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N 
2018-05-31 18:23:26.906856: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9949 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)

each time predict is called.

Can someone help me with this? Where am I wrong? when using GPU time taken should be less.

For installation I followed this link.

I am using Nvidia GeForce GTX 1080 Ti.

CPU is Intel(R) Core(TM) i7-7700K CPU

Model is MobileNet_v1

Upvotes: 2

Views: 1876

Answers (2)

Anon
Anon

Reputation: 1856

Maybe try to start=time.time() put this code after creating session (afterwith tf.Session(graph=graph) as sess:) for me creating session with gpu takes more time but makes predictions fast. Also have you ever tried with well known models I mean is that first time that your gpu gives bad performance?
Maybe try with VGG Nets you can find out benchmarks from here and compare to your gpu. if something seems to be wrong with your gpu focus on it but maybe that is about your model sometimes models gives much better performances on cpu

Upvotes: 1

zar3bski
zar3bski

Reputation: 3191

What king of GPU are you using. From what I understood, Tensorflow seems optimized for cuda -> Nvidia

Upvotes: 0

Related Questions