CODEWITHSUNDEEP
pythondictionarypersistent-object-store
Milla Well
Milla Well

Reputation: 3313

object storage in python that allow tuples as keys

I am searching for an object storage in python that allows me to store a dictionary having tuples as keys. I already tried shelve and shove, which both exit with an error as soon as I pass my dictionary. Are there any solutions out that provide this?

For shove, 

from shove import Shove
data = Shove('file://tmp')
("a",) in data

it gives me AttributeError: 'tuple' object has no attribute 'rstrip'. But only, if the tuple is not in data.

from shove import Shove
data = Shove('file://tmp')
data[("a",)] = 2
("a",) in data

would not throw an error.

For shelve,

import shelve
d = shelve.open('tmp/test.db')
d[('a',)] = 2

gives me TypeError: dbm mappings have string indices only

Upvotes: 0

Views: 769

Answers (2)

hBy2Py
hBy2Py

Reputation: 1832

Don't know how pythonic this is but... how about defining a constant separator string as something almost impossible to occur in your key strings:

sep = '#!#!#!#'

and then, when you need to create a key for shelve out of a tuple of strings, just .join them into a crude hash:

import shelve
d = shelve.open('tmp/test.db')
d[sep.join(('a',))] = 2

If you should need to regenerate a tuple key from information contained within the shelve repository, it's as easy as a .split:

my_dict = { tuple(k.split(sep)): d[k] for k in d.keys() }

Per here, this direct dict comprehension syntax is only supported for Python 2.7 and newer, but there are alternatives for 2.6 and earlier.

In your case, since you already have a dictionary defined, you'd have to do some dict-fu to hot-swap your current tuple keys for the str-ified hash when interacting with the shelve repository, but this shouldn't be too hard.

This approach isn't completely bug-free, but arguably can be made such that the probability of problems arising from collisions of sep with your tuple-of-str keys is vanishingly small. Also, note that this approach will only work if your keys are strictly tuples of strs.

Upvotes: 0

Serge Ballesta
Serge Ballesta

Reputation: 148890

shelve is a module from Python Standard Library. The doc is clear about that : the values (not the keys!) in a shelf can be essentially arbitrary Python objects — anything that the pickle module can handle ... The keys are ordinary strings

By construction shelve will only accept strings as keys.

Shove is still in Beta according to the documentation from pypi, and I could not see any evidence that it supports anything other that a string for the key (the error object has no attribute 'rstrip' let think it does not).

If I were you, I would stick to the well known shelve, and just wrap it with a key serialisation layer. As suggested by Padraic Cunningham, pickle should do the job.

Here is a (not extensively tested) possible implementation :

class tuple_dict(collections.MutableMapping):
    class iterator(collections.Iterator):
        def __init__(self, d):
            self.it = d.udict.__iter__()
        def __iter__(self):
            return self
        def next(self):
            return pickle.loads(next(self.it))
    def __init__(self, udict):
        self.udict = udict
    def __getitem__(self, key):
        ukey = pickle.dumps(key)
        return self.udict[ukey]
    def __setitem__(self, key, value):
        ukey = pickle.dumps(key)
        self.udict[ukey] = value
    def __delitem__(self, key):
        ukey = pickle.dumps(key)
        del self.udict[ukey]
    def keys(self):
        return [ pickle.loads(key) for key in self.udict.keys() ]
    def __iter__(self):
        return self.iterator(self)
    def __len__(self):
        return len(self.udict)
    def __contains__(self, key):
        return pickle.dumps(key) in self.udict
    def sync(self):
        self.udict.sync()
    def close(self):
        self.udict.close()

You would use it that way :

import shelve
underlying_d = shelve.open('tmp/test.db')
d = tuple_dict(underlying_d)

d will then accept tuple as keys and stores that all in the underlying shelf.

NB : if you later want to use a different persistence implementation, provided the implementation is a mapping (dict like class), you could reuse the tuple_dict by simply changing the close and sync methods (shelve specifice) but what would be needed by the other implentation. In fact apart from these 2 methods tuple_dict just wraps an ordinary dict - and as such any mapping class ...

Upvotes: 1

Related Questions