CODEWITHSUNDEEP
javascriptlistobject
Peter
Peter

Reputation: 11835

Best way to store a huge list with hashes in Javascript

I have a list with 10.000 entrys.

for example

myList = {};
myList[hashjh5j4h5j4h5j4]
myList[hashs54s5d4s5d4sd]
myList[hash5as465d45ad4d]
....

I dont use an array (0,1,2,3) because i can check very fast
-> if this hash exist or not.

if(typeof myObject[hashjh5j4h5j4h5j4] == 'undefined')
{
  alert('it is new'); 
}
else
{
  alert('old stuff'); 
}

But i am not sure, is this a good solution?
Is it maybe a problem to handle an object with 10.000 entries?

EDIT:
I try to build an rss feed reader which shows only new feeds. So i calculate an hash from the link (every news has an uniqe link) and store it in the object (mongoDB). BTW: 10.000 entrys is not the normal case (but it is possible)

Upvotes: 10

Views: 405

Answers (4)

Sky Kelsey
Sky Kelsey

Reputation: 19290

My advice:

  1. Use as small of a hash as possible for the task at hand. If you are dealing with hundreds of hashable strings, compared to billions, then your hash length can be relatively small.
  2. Store the hash as an integer, not a string, to avoid making it take less room than needed.
  3. Don't store as objects, just store them in a simple binary tree log2(keySize) deep.

Further thoughts:

  1. Can you come at this with a hybrid approach? Use hashes for recent feeds less than a month old, and don't bother showing items more than a month old. Store the hash and date together, and clean out old hashes each day?

Upvotes: 3

Martin Jespersen
Martin Jespersen

Reputation: 26183

It is not a problem in modern browser on modern computers in any way.

10k entries that take up 50 bytes each would still take up less than 500KB ram.

As long as the js is served gzipped then bandwidth is no problem - but do try to serve the data as late as possible so they don't block perceived pageload performance.

All in all, unless you wish to cater to cellphones then your solution is fine.

Upvotes: 0

Rusty Fausak
Rusty Fausak

Reputation: 7525

You can use the in operator:

if ('hashjh5j4h5j4h5j4' in myList) { .. }

However, this will also return true for members that are in the objects prototype chain:

Object.prototype.foo = function () {};
if ("foo" in myList) { /* will be true */ };

To fix this, you could use hasOwnProperty instead:

if (myList.hasOwnProperty('hashjh5j4h5j4h5j4')) { .. }

Whilst you yourself may not have added methods to Object.prototype, you cannot guarantee that other 3rd party libraries you use haven't; incidentally, extending Object.prototype is frowned upon, so you shouldn't really do it. Why?; because you shouldn't modify things you don't own.

Upvotes: 3

PiTheNumber
PiTheNumber

Reputation: 23542

10.000 is quite a lot. You may consider storing the hashes in a database and query it using ajax. It maybe takes a bit longer to query one hash but your page loads much faster.

Upvotes: 2

Related Questions