CODEWITHSUNDEEP
node.jstypescriptexpresscachingredis
pcace
pcace

Reputation: 640

sending data from redis (as cache) does not return the same data

i am trying to make a little express app which returns data from a postgres query. the query result should be cached in a Redis db. what I do is this:

app.get('/query_tile/:z/:x/:y', async (req: Request, res: Response) => {
    const client = new Client(connection);
    await client.connect();
    const setHeader = () => {
        res.set('Content-Encoding', 'br');
        res.set('Content-Type', 'application/x-protobuf');
        res.set('Vary', 'Origin, Access-Control-Request-Method, Access-Control-Request-Headers');
        res.set('Content-Disposition', `attachment; filename=tile_${z}_${x}_${y}.mvt`);
    }
    const { z, x, y } = req.params;
    const cacheKey = `tile_${z}_${x}_${y}`;
    const cachedData = await redis.get(cacheKey);

    if (cachedData) {
        
        setHeader()
        res.send(brotliCompressSync(cachedData));

    } else {

        const { rows } = await client.query('SELECT query_tile($1, $2, $3)', [z, x, y]);
        const data = rows[0];
        const compressedData = brotliCompressSync(data.query_tile);
        await redis.set(cacheKey, data.query_tile);
                
        setHeader()
        res.send(compressedData);
    }
    client.end()
});

the issue i am having is, that the cached result is not the same as the not cached one. what am i missing here?

both returns do have some data in it, but the one from the cache is not readable. at least not how i need it.

Thanks a lot for help!

EDIT: if i try to save the compressed data to the redis server like so:

    let cachedData = null;
    if (useCache) {
        cachedData = await redis.get(cacheKey);
    }

    if (cachedData) {
         const t1 = performance.now()
        console.log(`SENDING CACHED tile_${z}_${x}_${y}, took ${(t1 - t0) / 1000}s`);
        setHeader()
        res.send(cachedData);

    } else {
        const { rows } = await client.query('SELECT query_tile($1, $2, $3)', [z, x, y]);
        const data = rows[0];
        const compressedData = brotliCompressSync(data.query_tile);
        await redis.set(cacheKey, compressedData);
        
        const t1 = performance.now()
        console.log(`SENDING DB tile_${z}_${x}_${y}, took ${(t1 - t0) / 1000}s`);
        
        setHeader()
        res.send(compressedData);
    }

i do get a strange behaviour - the browser tells me that i should check my internetconnection enter image description here

but this only happens for chached tiles. If there is no cache available, it does download the tile properly.

Checking the redis server with "Redis Insight" it shows that there are my chached tiles in binary form. (but still it sais string on the left... is this supposed to be like that?) enter image description here

Upvotes: 0

Views: 142

Answers (1)

Guy Royse
Guy Royse

Reputation: 4312

Is the data you are caching binary data or does the brotliCompressSync function return plain text? Looks like binary data to me.

If that is correct, you need to use Buffers when you write binary data to Redis and a little bit of extra syntax when you read so it knows to return a Buffer.

By default, it will return ASCII-encoded text that looks something like this:

\xff\x00\xd8\xae\xa5x\x80J\xd2\x1c\x0c\x1ay\x89RZ\xd1\x1a\x029\xb7\x1a\xf3\xc5\xd4\x11s(...

Here's what that might look like in your code:

// setting binary data, assumes that brotliCompressSync returns a Buffer
const compressedData: Buffer = brotliCompressSync(data.query_tile);
await redis.set(cacheKey, compressedData);

// getting binary data
const cachedData = await redis.get(
  commandOptions({ returnBuffers: true }),
  cacheKey
);

Hope this helps!

Upvotes: 2

Related Questions