CODEWITHSUNDEEP
djangoredisdjango-redis
Kim Stacks
Kim Stacks

Reputation: 10812

Is it a good idea to use cache and how to use it for 4000-10000 data rows in Django for a single page?

Django 2.2

I need to fetch 4000-10000 data rows from a particular datatable (let's call this commonsites) amongst others to display a webpage.

I can narrow down to just 3 fields of these 4000-10000 rows (id, name, business_id)

My traffic is low. But i was wondering whether it's a good idea to use caching to fetch these 4000-10000 rows

The data for these rows are unlikely to change. But in case they do change or get deleted, how do I update/remove individual rows in the cache, rather than the entire cache?

Or is this even a good idea?

My installs are :

Update

Adding more clarity, the webpage is a retrieve. Once the page request is made, the javascript frontend will make an API call. This API call will then fetch these 4000-10000 data rows from a datatable.

So these datarows are pre-existing data.

The data is sent as API json data as a list in the json data.

Just to be clear, the data will not be paginated. It will all be displayed. And I haven't measured the data size, so I cannot say how large the data is. I doubt it be more than 5mb.

Upvotes: 1

Views: 672

Answers (1)

Ersoy
Ersoy

Reputation: 9586

Since we q/a in the comments, i may give a complete answer.

Keeping ~4K-10K of rows in a single string may not be a good practice. But you don't plan any pagination on the front-end side. Even you reduce the number of parameters to id, name, business_id - for that number of rows it may not be good for following reasons;

  • You need to get the value as whole, that may cause network problems. Especially when the traffic is getting high, every single person will fetch that chunk of data.
  • If you want to update/delete some of the rows, you need to (get+update+set)/set them as whole (network again)
  • You can't invalidate some part of the value.
  • You can't set partial TTL - you either expire/persist the whole.

The data for these rows are unlikely to change. But in case they do change or get deleted, how do I update/remove individual rows in the cache, rather than the entire cache?

Since you don't need pagination and want to keep them in a single key, instead of string you can use hash to satisfy conditions listed above.

127.0.0.1:6379> hset row 1 "some-json"
(integer) 1
127.0.0.1:6379> hset row 2 "some-json2"
(integer) 1
127.0.0.1:6379> hset row 3 "some-json3"
(integer) 1
127.0.0.1:6379> hset row 4 "some-json4"
(integer) 1
127.0.0.1:6379> hgetall row
1) "1"
2) "some-json"
3) "2"
4) "some-json2"
5) "3"
6) "some-json3"
7) "4"
8) "some-json4"
127.0.0.1:6379> hset row 3 "some-other-json"
(integer) 0
127.0.0.1:6379> hgetall row
1) "1"
2) "some-json"
3) "2"
4) "some-json2"
5) "3"
6) "some-other-json"
7) "4"
8) "some-json4"
127.0.0.1:6379> hdel row 3
(integer) 1
127.0.0.1:6379> hgetall row
1) "1"
2) "some-json"
3) "2"
4) "some-json2"
5) "4"
6) "some-json4"
127.0.0.1:6379>

You may use hset to update single/multiple row/s at once. You may delete via hdel an individual row from the cache. You can still get all the data as a single via using hgetall. There is no partial TTL for hash fields(sorted sets have that with some code at application layer) but hash is more suitable than strings for your use cases.

Upvotes: 1

Related Questions