CODEWITHSUNDEEP
cachingredis
mongy910
mongy910

Reputation: 497

Best caching strategy for repeat dictionary access with around 100,000 entries

I'm working on an application that provides translated subtitles for video content. My access pattern looks like this:

  1. Fetch video subtitles

  2. For each word, fetch translation from "translation database" (essentially just a dictionary of word -> word in x language)

  3. Return aggregated translations.

Each video may have thousands of words, and my current method takes multiple seconds for a single video - mainly taken up by the repeat fetching of word -> word translations. Since the entire translation dictionary is finite and relatively small, on the order of a few megabytes, I was thinking I could simply cache the entire dictionary to skip all of those tiny database fetches.

New to caching strategies - would something like Redis be overkill or would this be an intended use case? Would prefer to keep costs low. Using a Node.js backend if that is relevant.

Upvotes: 0

Views: 224

Answers (1)

Vikram Rawat
Vikram Rawat

Reputation: 1662

You have few choices:

  1. You can use Amazon Translate, which will do the heavy lifting for you. You can simply call an API and get tranlated text (both batch as well as real time APIs exist).

  2. If the cache size is only few megabytes then retrieve the dictionary from database as the application starts and keep it in application's memory. However, if dictionary is going to grow then save it in Redis.

Upvotes: 1

Related Questions