What makes MATLAB operations on vectors so fast?

Question

I had to write some MATLAB code, and at some point I had to apply a function I wrote elementwise to a vector. I considered two different way to do that:

1. Loop over the vector
2. Use MATLAB elementwise operations

In my case, I have a function group defined like:

function [g] = group(n, elm_per_group, n_groups)
    g = mod(n, n_groups);
    g(g > elm_per_group) = -1;
end

So that case #1 is like:

N = 1e4;
v = (1:N)';
w = zeros(1, N);
for i = 1 : N
    w(i) = group(v(i), elm_per_group, n_groups);
end

And case #2:

N = 1e4;
v = (1:N)';
w = group(v(:), elm_per_group, n_groups);

Of course it is absolutely not optimized to do that, except for the sake of the example.

I found very interesting things here and here, but I still can't understand what mechanisms MATLAB uses to speed up the computation, given an arbitrary function, which could even include randomness or have some kind of characteristic that will require specific calculation for each element of the vector.

I already have two ideas that explain, to me, part of the difference:

1. As MATLAB indexing is quite slow, so that looping is not a efficient way to reach every elements in the vector
2. The profiler tells that when looping over the vector, the group function has been called N times, whereas it is called only once when using MATLAB built-in. I expect that at least the time to manage the stack and the function call is shortened.

I'm not conviced that these two arguments are enough to explain the gap between the two approaches. To give some numbers, case #1 takes 7.2 ms and case #2 0.84 ms.

So, what are the additionnal mechanisms used by MATLAB to speed up the computation of N times the same function on the elements of a vector ?

(Note: please report spelling mistakes so that I can correct them.)

EDIT 1: changed name group to g to avoid name clash.

EDIT 2: modified the group function so that it is less bad and actually works.

Answer 1

MATLAB loops are quite fast. In fact, even vector-wise calculations are rarely faster than a loop. The problem is (as @Cris Luengo mentioned in the comments) the calling of (self-written) functions. I constructed a little example here (and fixed some issues in your code):

elm_per_group = 20;
n_groups = 10;
N = 1e7;
v = (1:N).';


%% function call
tic % start timing
% allocate memory
w1 = zeros(N,1);
% loop
for i = 1:N
    w1(i) = group(v(i), elm_per_group, n_groups);
end
toc % end timing


%% vectorwise
tic % start timing
w2 = group(v, elm_per_group, n_groups);
toc % end timing

%% direct looping
tic % start timing
% allocate memory
w3 = zeros(N,1);
% loop
for i = 1:N
    n = v(i);
    g = mod(n, elm_per_group);
    if g > n_groups
        w3(i) = -1;
    else
        w3(i) = g;
    end
end
toc % end timing


%% function call 2 (short-cut)
tic % start timing
% allocate memory
w4 = zeros(N,1);
% loop
for i = 1:N
    w4(i) = group2(v(i), elm_per_group, n_groups);
end
toc % end timing


%% check output
% if this assertion fails, not all results are the same!
assert( ~(any(w1-w2) || any(w1-w3) || any(w1-w4)) )



%% local function
function [g] = group(n, elm_per_group, n_groups)
    g = mod(n, elm_per_group);
    g( g > n_groups ) = -1;
end
function [g] = group2(n, elm_per_group, n_groups)
    g = mod(n, elm_per_group);
    if g > n_groups
        g = -1;
    end
end

Output

Elapsed time is 7.697662 seconds. % function call
Elapsed time is 0.463303 seconds. % vectorwise
Elapsed time is 0.669406 seconds. % direct looping
Elapsed time is 0.913067 seconds. % function call 2 (short-cut)

Not surprisingly, we observe (as you did) that your function-looping is very slow and that a vector-wise formulation (where I had to fix your code) is significantly faster. Now why is that so?

We can see from the other two examples that calling a function creates a 30% overhead but also that using an if-case can effectively save you time because it may avoid indexing and assigning a new value. It is a shortcut in your code.

Those are details and we are talking about milli-seconds here at 10.000.000 calls. But the take-away message is that unless you are not so mature in coding with MATLAB, take advantage of its native ability of vector-wise computation because it is fast! =)