Mismatch in results generated by CPU and GPU

Question

I was setting up to use Numba along with my AMD GPU. I started out with the most basic example available on their website, to calculate the value of Pi using the Monte-Carlo simulation.

I made some changes to the code so that it can run on GPU first and then on the CPU. By doing this, I just wanted to compare the time taken to execute the code and verify the results. Below is the code:

from numba import jit
import random
from timeit import default_timer as timer

@jit(nopython=True)
def monte_carlo_pi(nsamples):
    acc = 0
    for i in range(nsamples):
        x = random.random()
        y = random.random()
        if (x ** 2 + y ** 2) < 1.0:
            acc += 1
    return 4.0 * acc / nsamples

def monte_carlo_pi_cpu(nsamples):
    acc = 0
    for i in range(nsamples):
        x = random.random()
        y = random.random()
        if (x ** 2 + y ** 2) < 1.0:
            acc += 1
    return 4.0 * acc / nsamples

num = int(input())

start = timer()
random.seed(0)
print(monte_carlo_pi(num))
print("with gpu", timer()-start)

start = timer()
random.seed(0)
print(monte_carlo_pi_cpu(num))
print("without gpu", timer()-start)

I was expecting the GPU to perform better, and so it did. But however, some results for the CPU and the CPU were not matching.

1000000 # input parameter
3.140836 # gpu_out
with gpu 0.2317520289998356
3.14244 # cpu_out
without gpu 0.39849199899981613

I am aware that Python does not fare the long floating-point operations that well, but these are only 6 decimal places, and I was not expecting such a large discrepancy. Can anyone explain as to why this difference comes up?

JoshAdel · Accepted Answer

I've reorganized your code a bit:

import numpy 
from numba import jit
import random
from timeit import default_timer as timer

@jit(nopython=True)
def monte_carlo_pi(nsamples):
    random.seed(0)
    acc = 0
    for i in range(nsamples):
        x = random.random()
        y = random.random()
        if (x ** 2 + y ** 2) < 1.0:
            acc += 1
    return 4.0 * acc / nsamples


num = 1000000

# run the jitted code once to remove compile time from timing
monte_carlo_pi(10)

start = timer()
print(monte_carlo_pi(num))
print("jitted code", timer()-start)

start = timer()
print(monte_carlo_pi.py_func(num))
print("non-jitted", timer()-start)

results in:

3.140936
jitted code 0.01403845699996964
3.14244
non-jitted 0.39901430800000526

Note, you are not running the jitted code on your GPU. The code is compiled, but for your CPU. The reason for the difference in the computed value of Pi is likely due to differing implementations of the underlying random number generator. Numba isn't actually using Python's random module, but has its own implementation that is meant to mimic it. In fact, if you look at the source code, it appears as if the numba implementation is primarily designed based on numpy's random module, and then just aliases the random module from that, so if you swap out random.random for np.random.random, with the same seed, you get the same results:

@jit(nopython=True)
def monte_carlo_pi2(nsamples):
    np.random.seed(0)
    acc = 0
    for i in range(nsamples):
        x = np.random.random()
        y = np.random.random()
        if (x ** 2 + y ** 2) < 1.0:
            acc += 1
    return 4.0 * acc / nsamples

Results in:

3.140936
jitted code 0.013946142999998301
3.140936
non-jitted 0.9277294739999888

And just a few other notes:

When timing numba jitted functions, always run the function once to compile it before doing benchmarking so you don't include the one-time compile time cost in the timing
You can access the pure python version of a numba jitted function using .py_func, so you don't have to duplicate the code twice.

Mismatch in results generated by CPU and GPU

Answers (2)

What are the actual costs of doing so and where do they come from ?!?

EPILOGUE :

And the values ?

Related Questions