CODEWITHSUNDEEP
iphoneios
user317033
user317033

Reputation:

iPhone - 100k files of less than 100 bytes?

I have 100,000+ strings matched with a variable number of bytes 10-300. I will often do a lookup of a string to get those bytes.

My naive way to do this in the simplest/fastest manner was to save each string as a file and simply open the file and load the bytes. This won't work since the cluster size is 8k. Can you create a 'zip' of all 100,000 files without compression and would that be a simple and fast way to access the data? Do I have to learn how to do a database?

By simple I mean least code and fast I mean this is data is looked up often so must execute quickly. I really liked the file.open(str) idea.

EDIT - I don't have control of the strings except they will be unique. It looks like Tokyo Cabinet has a restrictive license, but I'm googling key-value store. Since yep I have no queries I just need to do a lookup and sqllite seems overkill.

NSDictionary - I didn't think of this, but I'm not sure I can load the whole thing into memory and keep it there since I don't control the total size.

Upvotes: 2

Views: 159

Answers (3)

Nick Veys
Nick Veys

Reputation: 23939

This sounds a lot like you just have a simple mapping of string -> data.

Something like this in some generic map notation:

data = {
  'string1' => 0x00000001,
  'string2' => 0x00000002,
  'string3' => 0x00000003,
}

It also seems like you don't need to 'query' per-se, but instead you know the key precisely and just need to get its associated value.

If this is the case, you could try a simple property list and see if it is performant enough. You can load that file and get back an NSDictionary object to use as you please. If it's not performant (I wouldn't be surprised), a sqlite database or some small embedded key-value store should do the trick.

Upvotes: 0

Mundi
Mundi

Reputation: 80265

Sounds like you want to use core data. This would be infinitely faster than having Ks of files. My guess is that this is the fastest and most robust solution.

Upvotes: 4

pronvit
pronvit

Reputation: 4289

Use simple fast nosql database like Tokyo Cabinet. It's very easy to setup/use and will be faster than core data for your needs (you have no complex queries) and much faster than files.

Upvotes: 5

Related Questions