CODEWITHSUNDEEP
python
Eyal Ch
Eyal Ch

Reputation: 10066

load variables from in memory setting file

i have settings.py file that looks like:

DEBUG = True
CACHE_IP = '0.0.0.0'
DB_SETTINGS = {
    'SERVERS': ['127.0.0.1:80'],
    'MAX_IDLE_TIME' : 180,
}

this file is stored at external cloud storage.

I have a process the get the url to this file, and load it into memory (not to disk). when loading it into memory it will be string type and looks like:

"DEBUG = True\n CACHE_IP = '0.0.0.0'\n DB_SETTINGS = {\n    'SERVERS': ['127.0.0.1:80'],\n    'MAX_IDLE_TIME' : 180,\n}\n"

How for example i can get the values from this string? i mean, how can i get a dictionary where key=verb name from file, and value = value from file? is there any pythonic way for this?

so my dictionary will look like:

{'DEBUG': True, 'CACHE_IP' = '0.0.0.0', 'DB_SETTINGS': {...}}

Upvotes: 0

Views: 397

Answers (2)

Amit Tripathi
Amit Tripathi

Reputation: 7261

Your requirement is very specific but this kind of requirements do arise. So, I tried to solve this using method below:

In [102]: import ast # Do not use eval(its unsafe)

In [103]: string = "DEBUG = True\n CACHE_IP = '0.0.0.0'\n DB_SETTINGS = {\n    '
     ...: SERVERS': ['127.0.0.1:80'],\n    'MAX_IDLE_TIME' : 180,\n}\n"
     ...:

In [104]: def parse_key_val(string):
     ...:     tmp_dict = {}
     ...:     for r in re.split('(?<![{},])\n (?!{)', string):
     ...:         key, val = r.split(" = ")
     ...:         if '{' in val:
     ...:             val = ast.literal_eval(val)
     ...:         tmp_dict[key] = val
     ...:     return tmp_dict
     ...:

In [105]: parse_key_val(string)
Out[105]:
{'CACHE_IP': "'0.0.0.0'",
 'DB_SETTINGS': {'MAX_IDLE_TIME': 180, 'SERVERS': ['127.0.0.1:80']},
 'DEBUG': 'True'}

This solution works great for the string you provided but it might(or not) fail in some complex cases. But this solution gives a good starting point of how to start. You might need to make some modifications to make it work for complex cases.

Now coming to the program, the important parts here to understand are:

re.split('(?<![{},])\n (?!{)', string)

and 

val = ast.literal_eval(val)

The pattern (?<![{},])\n (?!{) in re.split means, split the string on character \n but only if it is not after { or } or , and not before {. This regex is created to not split \n line character in dict strings.

Not the second part val = ast.literal_eval(val) builds on top of the above regex step as above step doesn't break the dict string. The resulting dict part of string after split is:

"DB_SETTINGS = {\n    'SERVERS': ['127.0.0.1:80'],\n    'MAX_IDLE_TIME' : 180,\n}\n"

With ast.literal_eval the task of parsing dict, nested dict and list inside it very easy. That it!

One more thing, do not use eval as one of the solution suggests here, it's a security risk.

You can read more about ast here

Upvotes: 2

Ram
Ram

Reputation: 51

One way is to use the eval function to execute the text. 

eval("DEBUG = True\n CACHE_IP = '0.0.0.0'\n DB_SETTINGS = {\n    'SERVERS':       ['127.0.0.1:80'],\n    'MAX_IDLE_TIME' : 180,\n}\n")
settings_dict = {}
settings_dict['DEBUG'] = DEBUG
settings_dict['CACHE_IP'] = DEBUG
settings_dict['DB_SETTINGS'] = DB_SETTINGS

I don't know of a way to 'automate' the dictionary creation.

Upvotes: 1

Related Questions