CODEWITHSUNDEEP
livecode
MadPink
MadPink

Reputation: 294

How much information can be stored in an array before performance degrades?

for simplicity, say we have an array of personal contacts, the first key is the contact name, and there are subkeys which include phone number, address, email, notes and date of birth

how many contacts could be held in an array in memory without running into performance issues?

for reference, this would be running on older machines with 512MB of RAM running Debian Linux, so resources are scarce

Upvotes: 2

Views: 580

Answers (2)

Mark
Mark

Reputation: 2435

You won't find much slow-down as arrays increase. In an ideal situation, an array doesn't slow down because the amount of data increases; it only slows down by adding millions of keys.

In other words, if your array has a few gigabytes of data and your computer is a 64 bit machine with sufficient memory, your computer simply won't slow down at all. If you have a 32 bit machine and you're loading more than 4 GB data, it will make heavy use of virtual memory and you will see significant slow down.

As your array has more keys, the search routine that finds the right key may need a little more time, but as long as you have less than a few billions contacts in your database (I would assume 2^32 contacts but I didn't check the exact number) I would expect any slow-down to be of an acceptable magnitude.

However, since you indicate to be using older machines with 512MB ram, the size of the data is likely to become a problem. The biggest problem will be that eventually you may have a data stack bigger than the size of available memory. Debian will take about half of the physical memory, which means that you have the other half of 256MB available for other apps, including your contacts database. If your app uses a nice GUI, your app will quickly need more than 256MB of memory and it is going to rely heavily on virtual memory, causing it to slow down.

Additionally, arrays aren't always the best solution. First of all, you are probably better off using SQLite. SQLite has very fast specialized search routines, which are fast than any LiveCode routine using arrays. Second, a repeat for each line loop using a simple list is sometimes faster than a repeat for each key loop, because for each lineaccess the data directly while repeat for each key first loops through the keys and then still needs to access the array to check what is in the element of that key.

Upvotes: 1

makeshyft_tom
makeshyft_tom

Reputation: 174

I'd be interested in knowing the answer to this as well..I was however going to do some testing myself on my app ..... its really easy to populate an array automatically.

global MyTestArray

repeat with x = 1 to 1000000
put uuid("random") into MyTestArray[x]["key1"]
put uuid("random") into MyTestArray[x]["key2"]
end repeat

to time your handlers use:

on TimedHandler

local start_time

put the milliseconds into start_time

// your handler that reads the array

put the milliseconds into end_time
answer (the milliseconds - start_time) / 1000

end TimedHandler

Upvotes: 2

Related Questions