CODEWITHSUNDEEP
memoryluagarbage-collection
bokre369
bokre369

Reputation: 1

Lua Memory and Garbage Collection Testing

I have been running some tests to see how Lua Memory and it's Garbage Collection function works. I am working with a very limited amount of RAM so I wanted to get a good idea of the best methods of memory management. I am using Lua 5.3.

I have been running the following code to watch the memory grow until it hits a 'not enough memory' error:

x = {}
table.insert(x, "111111111111111111111111111111111111111111111111111111111111")
while(1) do
    table.insert(x, "111111111111111111111111111111111111111111111111111111111111")
    print(string.format("Main: LUA memory: =  %s", collectgarbage("count")))
    socketlib.sleep(0.1)
end

I have run this with the default garbage collect settings as well as stopping the garbage collection from running by using collectgarbage("stop"). Below is a plot of the data that I collected, memory over time.

Memory vs Time:

1

The start of the graph is when I began the program and the end is when it inevitably runs into the memory error and stops. The beginning of both methods are nearly, if not exactly, identical and they both bomb out around the same time as well.

I find a few things suspicious here.

First is that when the GC is turned off, once it hits around 900 kB I see a drop down to a little over 300 kB for the collectgarbage("count"). It then runs for close to another 100 kB before bombing out. The amount of memory used seems to be around my limit of 1 MB if we don't count the drop, but I find this behavior odd.

Secondly, when garbage collection is left on by default, the memory does not get up to where I would expect before it runs out of memory. It looks like it is collecting based on the data from collectgarbage("count") but does not seem to reflect that.

My bigger concern here, which is what my actual question on this post is, can anyone tell me why the memory appears to jump up every so often? I would expect a fairly flat line of memory increases, maybe a few slightly different sized jumps, but not to the degree that I am seeing. I am not asking about the large drop off when garbage collect is stopped, I am asking about the smaller jumps up leading up to that.

Another semi-suspicious point is that whenever the memory jumps up, it appears to be around size 1kB, 2kB, 4kB, 8kB, 16kB, then 32kB.

Any help would be greatly appreciated!

Upvotes: 0

Views: 1110

Answers (1)

Luatic
Luatic

Reputation: 11191

My bigger concern here, which is what my actual question on this post is, can anyone tell me why the memory appears to jump up every so often? [...] Another semi-suspicious point is that whenever the memory jumps up, it appears to be around size 1kB, 2kB, 4kB, 8kB, 16kB, then 32kB.

My guess would be that this is Lua extending the size of the array part of the table. This is usually done by doubling the size, hence the powers of two in memory consumption. It is required for table.insert to run in amortized constant time: If it had to extend the array part with each operation, table.insert would run in linear time, which is usually unacceptable.

Note that the string isn't stored multiple times due to string interning, so all entries in your table are actually just references to the same string; you'd get roughly the same memory consumption if you just inserted numbers into the table.

Upvotes: 2

Related Questions