CODEWITHSUNDEEP
pythoncouchdblarge-datanosql
The Unfun Cat
The Unfun Cat

Reputation: 32028

NOSQL databases for sets (as opposed to dicts/key-value stores)?

I'm trying to find a database to function as a Python set. This is because my data is way too big to be stored in memory.

I tried using SQLite, but heard that it might have performance issues for > ten gigabytes of data, so I'm looking at trying CouchDB

Problem is that it seems to work like a dict, not like a set?

Is there a database tool that functions as a Python set? That is, it just stores values and not key-value pairs?

(I have to code in Python so I'm interested in something that is easy to use with Python)

Edit:

I will store it as one giant set, not several small ones.

Upvotes: 2

Views: 1018

Answers (3)

ekini
ekini

Reputation: 100

Why don't you make a collection with set values used as unique key?

UPD: for example, you have document like this:

{
    _id: "someid",
    youset: {val1, val2, val3},
}

You can create a new collection like:

{
    _id: val1,
    owner: "someid"
}
{
    _id: val2,
    owner: "someid"
}
{
    _id: val3,
    owner: "someid"
}
...

Since you don't need whole data at the same time, there is no need to embed it inside main document.

Upvotes: 0

acjay
acjay

Reputation: 36691

A key/value store acts like a dict, but that's pretty much how set is implemented anyway, according to the main answer of How is set() implemented?. Why not just use a small dummy value, and do your set operations on the keys?

Upvotes: 1

jdi
jdi

Reputation: 92657

Redis can store Set data types:
http://redis.io/topics/data-types

It has a python client.

Upvotes: 1

Related Questions