The rules for the # operator

Question

I am trying to understand the rules for the # operator.

Consider the following examples:

> t1 = table.pack(nil, 1, nil, 2)
> #t1
4
> for k, v in pairs(t1) do print(k, v) end
2   1
4   2
n   4
> t2 = table.pack(nil, 1, nil, 2, nil)
> #t2
2
> for k, v in pairs(t2) do print(k, v) end
2   1
4   2
n   5
> t3 = table.pack(nil, 1, nil, 2, nil, 3, nil)
> #t3
0
> for k, v in pairs(t3) do print(k, v) end
2   1
4   2
6   3
n   7

I cannot make heads or tails of this. The value of the n field consistently matches the number of arguments originally passed to table.pack, but the value returned by the # operator is all over the place, as shown in the summary below:

| n | # |
|---+---|
| 4 | 4 |
| 5 | 2 |
| 7 | 0 |

What is the rule for #? In other words, if I see the output from

for k, v in pairs(sometable) do print(k, v) end

...how can I deduce the value returned by #sometable?

(In case it matters, I am using lua5.3.)

---

EDIT: I should add that, for all tables t1, t2, and t3 in the example above, ipairs returns nothing. E.g.

> for i, v in ipairs(t0) do print(i, v) end
> -- no output

Answer 1

Arrays and nils in Lua are a complex topic.

When you use table.pack, it counts the number of arguments and sets the tables n value to that. This is not the case if you use a table constructor like {1, nil, 3, nil}.

The # operator, however, is does something slightly different.

From the manual:

The length operator applied on a table returns a border in that table. A border in a table t is any natural [...] index in a table where a non-nil value is followed by a nil value (or zero, when index 1 is nil).

What that means is that, for the table

t = {1, nil, 3, nil}

it could return either 1, because t[1] is 1 and t[2] is nil, or 3, because t[3] is 3 and t[4] is nil.

The ipairs() function does yet another different thing: it actually counts up indices from 1, until it hits a nil in the table, so it will always count up to the first border in the table.

If you want the # operator to return the n value of the table, in Lua 5.2 or newer you can do the following:

local t = {n=10}
setmetatable(t, {__len=function(self) return self.n end})
print(#t) -- Will print 10, even though t has 0 numeric indices

Note, however, that this will not work in LuaJIT, which is based on Lua 5.1.

Similarily, you can set the __ipairs metamethod of a table t so ipairs(t) counts from 1 to t.n instead of to the first nil element.

---

EDIT: As for why ipairs does nothing for your examples, this mostly results from what I already explained about ipairs, but since the first key in the table is nil, it instantly assumes it's empty and thus does nothing.

---

This is not really relevant, as you should not rely on this kind of implementation-specific behavior, but here's how PUC Lua 5.3 implements the # operator for

/*
** Try to find a boundary in table 't'. A 'boundary' is an integer index
** such that t[i] is non-nil and t[i+1] is nil (and 0 if t[1] is nil).
*/
lua_Unsigned luaH_getn (Table *t) {
  unsigned int j = t->sizearray;
  if (j > 0 && ttisnil(&t->array[j - 1])) {
    /* there is a boundary in the array part: (binary) search for it */
    unsigned int i = 0;
    while (j - i > 1) {
      unsigned int m = (i+j)/2;
      if (ttisnil(&t->array[m - 1])) j = m;
      else i = m;
    }
    return i;
  }
  /* else must find a boundary in hash part */
  else if (isdummy(t))  /* hash part is empty? */
    return j;  /* that is easy... */
  else return unbound_search(t, j);
}

As you can see, Lua uses binary search to find the boundary, which only makes sense if you assume that there's only one; otherwise it might just skip the first one at random, yet suddenly find it just by adding one value to the table.

Also keep in mind here that the array part of a table doesn't increase every time you add a value. If I remember correctly, it is always doubled; so you can have a 4 element table, add one element and Lua will increase the array size to 8. Add another 4 elements, and it will grow to 16, even though it only holds 9 elements. This is to avoid unnecessary allocations.

Answer 2

I asked a similar question

To summarise: It will count until one of non-nil elements, since Lua does not support storing nils in tables. You'll need to count manually using ipairs instead. Here is the reference