CODEWITHSUNDEEP
pythondjangocaching
mp85
mp85

Reputation: 422

Django - Looking for a good way to cache a dictionary

In a Django project I'm working on, the user can search for points of interest by city. The area where the search can happen is relatively small (approx. 400-500 cities), and I have a dictionary that has records for about 300 of them.

Up until now, every time a user searchs for a city, I look into the dictionary to see if the city is there, if not I just ask Google's geocoder for the location (via the geopy library).

Since the lookup area is small, Google's geocode doesn't behave really well; it's really easy to screw up the search if you even mess up one character, and I had to put a lot of checks to make sure the geocode phase won't break.

I wanted to change this behaviour and rely less and less on the geocoders, by adding a city to the dictionary if it wasn't in the dictionary before. So the dictionary would have to be pickled, saved into a file and then loaded, updated (via the geocoded data) and eventually saved again once a new city is encountered.

I'm a bit worried about the performances, does this behaviour means trouble (at least in the first period, when it's likely that a new city isn't found in the dictionary)? Should I cache the dictionary in another way or even take a completely different approach?

Any opinion would be of great help, thanks.

Upvotes: 0

Views: 1442

Answers (1)

Kedar
Kedar

Reputation: 1698

Reading a file would work, but it will become slow as the number of requests per second start to increase.

Consider using Redis. It is a supercharged dictionary and cache that you can use to store key-value pairs and more (set, lists, strings, hashes). It runs primarily in memory but can also be stored to disk and reloaded.

For your case, every time a user searches for a city, you check Redis for key (city name as string or ID, however you wish), if it exists, use whatever value (co-ordinates?) it has for that city key, otherwise fall-back to geocoder and add a new key to Redis with its the result from geocoder.

There is also a django app for Redis to make it easier: https://niwibe.github.io/django-redis/

Upvotes: 1

Related Questions