CODEWITHSUNDEEP
pythondata-structurespersistence
tMC
tMC

Reputation: 19315

Python nested data persistance

I need a way to store system configuration data and i've found it easy to understand / comprehend with nested classes:

>>> class syscnf:
...     class netwrk:
...         class iface:
...             class eth0:
...                 address = '192.168.0.100'
...                 netmask = '255.255.255.0'
...                 mtu = 1500
...             class eth1:
...                 address = '172.25.0.23'
...                 netmask = '255.255.255.128'
...         class route:
...             default = '192.168.0.1'
>>>
>>> print syscnf.netwrk.iface.eth0.address
192.168.0.100
>>>

but this structure can't be pickled and saved. I know I could shelve it in key/value pairs:

syscnf.netwrk.iface.eth0.address => 192.168.0.100
syscnf.netwrk.iface.eth0.netmask => 255.255.255.0
syscnf.netwrk.route.default => 192.168.0.1
syscnf.... etc

but that seems like it would be hard to manage and prone to errors?

or i could save it in an sqlite DB, but then i would need a new table, and schema for each final level of config data and storing pickled data in sqlite seems like it would hard to manage.

I need a way to persist this data on an embedded platform, so it needs to relay on pure python and included modules, (or be really easy to cross compile- i haven't tried but ZODB doent read like it would be easy to cross- compile etc)

What have you used that was flexible and straight forward? It doesn't need to be high performance, and concurrency would be nice, but not 'required'

I've never had to do anything like this and was hoping you guys had some insight / experience you'd like to share!

Upvotes: 1

Views: 190

Answers (2)

Domingo Ignacio
Domingo Ignacio

Reputation: 1214

I agree with Ignacio Vazquez-Abrams that using a JSON-like structure would make it a lot easier to work with, meaning you might instead have a structure like:

syscnf = {
  'netwrk': {
    'iface': {
       'eth0': {
          'address': '192.168.0.100',
          'netmask': '255.255.255.0',
          'mtu': 1500,
        }
        # ...
     }
     'route': {
        'default': '192.168.0.1',
     }
  }
}

That is, mostly dictionaries within dictionaries (and other values like strings, numbers, and lists). And then you'd need to access the elements like dictionaries, not classes, e.g,

print syscnf['netwrk']['iface']['eth0']['address']

instead of: 

print syscnf.netwrk.iface.eth0.address

Then you can just use the json or simplejson module (or even good old pickle/cPickle) to serialize.

Of course, you do lose some prettiness, and gain a bunch of brackets. If that's important to you, then you could try something like YAML (Python module available), or you could keep what you have and hand-write a converter that recursively replaces a class with the dictionary keyed by the memebers of its dir(), removing stuff like doc and module.

E.g.,

from types import ClassType
def jsonize_class( klass ):
  def recurse( child ):
    return jsonize_class(child) if isinstance(child, ClassType) else child
  def entry( key ):
    return key, recurse( getattr(klass,key) )
  return dict(( entry(key) for key in dir(klass) if not key.startswith('__') ))

Which will then convert a class in the format you're already using into a json-like structure:

>>> class bob:
...     x = 9
...     y = "hello"
...     class job:
...             bill = 999
...     class rob:
...             pass
... 
>>> jsonize_class(bob)
{'y': 'hello', 'x': 9, 'job': {'bill': 999}, 'rob': {}}

Then to take the serialized JSON object and make it accessible in the style you like, you could reverse the process:

from types import DictType
def classize_json( the_json ):
  if isinstance( the_json, DictType ):
    class returned_class: pass
    for key, val in the_json.iteritems():
      setattr( returned_class, key, classize_json(val) )
    return returned_class
  else:
    return the_json

E.g.:

>>> jBob = jsonize_class(bob)
>>> jBob
{'y': 'hello', 'x': 9, 'job': {'bill': 999}, 'jimmy': 99, 'rob': {}}
>>> cBob = classize_json(jBob)
>>> cBob.y
'hello'
>>> cBob.job.bill
999
>>> cBob.jimmy
99

Upvotes: 3

Ignacio Vazquez-Abrams
Ignacio Vazquez-Abrams

Reputation: 798546

All the cool kids use json. Also, having your structure use instances instead of bare classes will make it much easier to serialize.

Upvotes: 4

Related Questions