CODEWITHSUNDEEP
djangomemcached
Alex
Alex

Reputation: 18522

Apparent bug storing large keys in django memcached

I've been following instructions on setting up memcached to support large values.

The use case is basically that I want to keep a fairly large machine-learning model in memory, and use it to respond to queries. The size of the model might vary between 0.5 and 100 MB.

These are the custom keys in my /etc/memcached.conf

# custom flags
-I 10M
-m 200
-vvv

Here's the keys set after restarting memcached:

> sudo service memcached restart
Restarting memcached: memcached.
> echo "stats settings" | nc localhost 11211
STAT maxbytes 67108864
STAT maxconns 1024
STAT tcpport 11211
STAT udpport 11211
STAT inter 127.0.0.1
STAT verbosity 3
STAT oldest 0
STAT evictions on
STAT domain_socket NULL
STAT umask 700
STAT growth_factor 1.25
STAT chunk_size 48
STAT num_threads 4
STAT num_threads_per_udp 4
STAT stat_key_prefix :
STAT detail_enabled no
STAT reqs_per_event 20
STAT cas_enabled yes
STAT tcp_backlog 1024
STAT binding_protocol auto-negotiate
STAT auth_enabled_sasl no
STAT item_size_max 10485760
STAT maxconns_fast no
STAT hashpower_init 0
STAT slab_reassign no
STAT slab_automove no
END

So there appears to be good news and bad news: ITEMSIZEMAX has changed to about 10 MB, but maxbytes has remained at default 67 MB.

The real problem is that I still can't store keys > 1 MB!

If I run the following Python script, it fails when trying to store a numpy array of size ~0.4 MB

from django.core.management import setup_environ
from myapp import settings
setup_environ(settings)

key = "test_key"

def store_test(val):
 cache.set(key, val)
 if cache.get(key) is not None:
  cache.delete(key)
  return True
 else:
  return False

from django.core.cache import cache
from numpy import arange

i = 10
while i <= 1000000:
 yes = store_test(arange(i))
 if yes:
        print "stored", i, "successfully"
 else:
        print "failed to store", i
        print "bytes:", arange(i).nbytes
 i *= 10

Of course, to run this code, you will need to replace 'myapp' with a valid Django module that contains a settings.py file.

Can anyone replicate the problem, help me debug this, or suggest a solution?

** EDIT **

I wrote a bash script to test memcached directly instead of going through Python, and oddly enough it seems to work!

for i in 10 1000 10000 100000 1000000 2000000 10000000 11000000
do
        bytes=$(yes | head -n $i | tr -d '\n')
        echo doing $i bytes
        (echo delete key; echo set key 0 900 $i; echo $bytes; echo get key; sleep 1) | telnet localhost 11211 > output_$i
        outputsize=$(stat -c%s "output_$i")
        echo output size is $outputsize - should be about $(($i+110))
done
echo delete key | telnet localhost 11211
~

Upvotes: 1

Views: 639

Answers (1)

Alex
Alex

Reputation: 18522

So I couldn't seem to find it before, but apparently this question has already been answered.

Django caching a large list

The problem is that both memcached and Django's memcached binding set limits of 1 MB. I'm pretty sure this is a bug - the python binding should respect memcached's actual configuration. The new value has to be hardwired in, apparently.

Upvotes: 2

Related Questions