CODEWITHSUNDEEP
python
J.P.
J.P.

Reputation: 141

Using a dictionary of dictionaries to store constants

I'm currently in the middle of making a web app in Python. It will contain and use a lot of unchanging data about several things. This data will be used in most of the app to varying degrees.

My original plan was to put this data in a database with all the other changing data, but to me this seems excessive and a potential and unnecessary choke point (as the same data will be queried multiple times / in various combinations on most page loads / interactions).

I've rewritten it so that the data is now stored in several dictionaries of dictionaries (i.e. in memory), essentially being constants, from which the data is accessed through functions. The data looks a bit like this:

{
    0: {
           'some_key': 'some_value',
           'another_key': 'another_value',
           ...
    },
    ...
}

Is this memory efficient? Is there a more tried and true / pythonic / just plain better (in terms of speed, memory use, etc.) way of doing this kind of thing? Is using a database actually the best way of doing it?

Upvotes: 1

Views: 231

Answers (2)

Marcin
Marcin

Reputation: 49816

There's nothing especially wrong with this approach, but I'll note some issues:

  1. Why nested dictionaries? Why not a flat dict, or even a module filled with variables?

  2. If these are "objecty" data, why not store them in actual objects? Again, these could live in variables in a module, or in a dict.

  3. Your web framework may already have a solution for this specific problem.

Upvotes: 4

ChrisProsser
ChrisProsser

Reputation: 13088

This seems like a perfectly sensible way of storing your reference data so long as you have plenty of memory for the data that you need to hold. I agree that this should be quicker than reading from a database as the data will already be in memory and sorted for efficient access.

If you don't want to store this data actually in your source code you could store it in a json file (json.dump() to write out to a file and json.load() to read back in). But you would want to read this into memory at the point of the application starting up and then just keep it in memory rather than going back to the file for it every time.

Upvotes: 0

Related Questions