CODEWITHSUNDEEP
python
NewGuy
NewGuy

Reputation: 3423

More pythonic way to replace keywords in a string?

I am attempting to wrap an API with the following function. The API has end points that look similar to this:

/users/{ids}
/users/{ids}/permissions

The idea is that I'll be able to pass a dictionary to my function that contains a list of ids and those will be formatted as the API expects:

users = {'ids': [1, 2, 3, 5]}
call_api('/users/{ids}/permissions', users)

Then in call_api, I currently do something like this

def call_api(url, data):
    for k, value in data.items():
        if "{" + key + "}" in url:      
            url = url.replace("{"+k+"}", ';'.join(str(x) for x in value))
            data.pop(k, None)

This works, but I can't imagine that if statement is efficient.

How can I improve it and have it work in both Python 2.7 and Python 3.5?

I've also been told that changing the dictionary while iterating is bad, but in my tests I've never had an issue. I am poping the value, because I later check if there are unexpected parameters (ie. anything left in data). Is what I'm doing now the right way?

Upvotes: 2

Views: 141

Answers (4)

wegry
wegry

Reputation: 8147

Instead of modifying a dictionary as you iterate over it, creating another object to hold the unused keys is probably the way to go. In Python 3.4+, at least, removing keys during iteration will raise a RuntimeError: dictionary changed size during iteration.

def call_api(url, data):
    unused_keys = set()
    for k, value in data.items():
        key_pattern = "{" + k + "}"
        if key_pattern in url:
            formatted_value = ';'.join(map(str, value))     
            url = url.replace(key_pattern, formatted_value)
        else:
            unused_keys.add(k)

Also, if you think that you're more likely to run into an unused key, reversing the conditions might be the way to go.

Upvotes: 1

Jared Goguen
Jared Goguen

Reputation: 9010

The formatting keys can be found with a RegEx and then compared to the keys in the dictionary. Your string is already setup to use str.format, so you apply a transformation to the values in data, and then apply that transformation.

import re
from toolz import valmap

def call_api(url, data):
    unused = set(data) - set(re.findall('\{(\w+)\}', url))
    url = url.format_map(valmap(lambda v: ';'.join(map(str, v)), data))
    return url, unused

The usage looks like:

users = {'ids': [1, 2, 3, 5], 'unused_key': 'value'}
print(call_api('/users/{ids}/permissions', users))
# ('/users/1;2;3;5/permissions', {'unused_key'})

This isn't going to time that well, but it's concise. As noted in one of the comments, it seems unlikely that this method is be a bottleneck.

Upvotes: 1

ekhumoro
ekhumoro

Reputation: 120678

You could use re.subn which returns the number of replacements made:

import re

def call_api(url, data):
    for k, value in list(data.items()):
        url, n = re.subn(r'\{%s\}' % k, ';'.join(str(x) for x in value), url)
        if n:
            del data[k]

Note that for compatibilty with both python2 and python3, it is also necessary to create a copy of the list of items when destructively iterating over the dict.

EDIT:

It seems the main bottleneck is checking that the key is in the url. The in operator is easily the most efficient way to do this, and is much faster than a regex for the simple pattern that is being used here. Recording the unused keys separately is also more efficient than destructive iteration, but it doesn't make as much difference (relatively speaking).

So: there's not much wrong with the original solution, but the one given by @wegry is the most efficient.

Upvotes: 1

Bharel
Bharel

Reputation: 26994

Here is the way to do it. First, the string is parsed for the keys. It then remembers all keys not used in the url and saves it in the side. Lastly, it formats the url with the given parameters of the dict. The function returns the unused variables and the formatted url. If you wish you can remove the unused variables from the dict by iterating over them and deleting from the dict. Here's some documentation with examples regarding the format syntax.

import string

users = {'ids': [1, 2, 3, 5]}

def call_api(url, data):
    data_set = set(data)
    formatter = string.Formatter()
    used_set = {f[1] for f in formatter.parse(url) if f[1] is not None}
    unused_set = data_set - used_set
    formatted = url.format(**{k: ";".join(str(x) for x in v) 
                              for k, v in data.items()})
    return unused_set, formatted

print(call_api('/users/{ids}/permissions', users))

Upvotes: 1

Related Questions