Memcached item overhead

Question

Does anybody know how much overhead per single item is required? When I insert 1 byte key the 'stats' command shows 65 bytes being consumed per item.

Is this expected and fixed?

Thanks, Piotr

Answer 1

Yes, Memcache's own data structures eat up more than 50 byte per item. It depends a little on your data and key, so assume 60+ bytes when on a 64 bit machine.

One can see it when looking at memcache's code: https://github.com/memcached/memcached/blob/master/memcached.h

Here's what makes up an item:

/**
 * Structure for storing items within memcached.
 */
typedef struct _stritem {
    struct _stritem *next;
    struct _stritem *prev;
    struct _stritem *h_next;    /* hash chain next */
    rel_time_t      time;       /* least recent access */
    rel_time_t      exptime;    /* expire time */
    int             nbytes;     /* size of data */
    unsigned short  refcount;
    uint8_t         nsuffix;    /* length of flags-and-length string */
    uint8_t         it_flags;   /* ITEM_* above */
    uint8_t         slabs_clsid;/* which slab class we're in */
    uint8_t         nkey;       /* key length, w/terminating null and padding */
    /* this odd type prevents type-punning issues when we do
     * the little shuffle to save space when not using CAS. */
    union {
        uint64_t cas;
        char end;
    } data[];
    /* if it_flags & ITEM_CAS we have 8 bytes CAS */
    /* then null-terminated key */
    /* then " flags length\r\n" (no terminating null) */
    /* then data with terminating \r\n (no terminating null; it's binary!) */
} item;

There are 3 pointers, which on a 64 bit machine sum up to 3 * 8 = 24 byte. There there a two time stamps, which should be 32 bit, so they make 8 byte in total. the following int should be 8 byte, the short should be 2, and u_int8 should be a single byte, so they sum up to 14.

This makes a total of 46 bytes.

If CAS is enable there is a 64 bit CAS value, which adds up 8 bytes: 54 Byte in total

Now it gets messy: Following is character data, where flags and length of data are printed as decimals (code can be found at https://github.com/memcached/memcached/blob/master/items.c, line 80). Given that flag is "0", key is one char, and data is empty, this makes:

• 1 Byte key + 1 Byte 0
• 1 Byte "Space", 1 Byte flag "0", 1 Byte "Space", 1 Byte "0"
• 2 Byte "\r\n" ( a line feed)
• 2 Byte for termination data

That's another 10 Byte, making a total of 64 Byte for the smallest possible memcache item size (CAS enabled, you may disable it using the -C option).

If you get 65 Byte, maybe your client set a flag of "10" or such.

Answer 2

It varies by item class, and can be configured to better suit your application. Your best bet is to store lots of things and monitor your cache rate. If the application performance is suffering, then you can adjust.