Using Redis Pattern Subscribe

Question

I am working on a NodeJS application.

I am new to redis, I just installed it yesterday, but I'd like to be able publish this data and subscribe to it from another process.

Suppose I have the following data:

    var Exchanges = [
        {
            _id: 'tsx',
            name: 'Toronto Stock Exchange',
            data: {
                instrument: [
                    {
                        symbol: 'MBT'
                        markPrice: 0,
                    }, 
                    {
                        symbol: 'ACQ'
                        markPrice: 0,
                    }
                ],
                orderBooks: [
                    {
                        symbol: 'MBT',
                        bids: [],
                        asks: [],
                    },
                    {
                        symbol: 'ACQ',
                        bids: [],
                        asks: [],
                    }
                ],
                trades: [
                    {
                        timestamp: "2014-11-06T20:53:00.000Z",
                        symbol: "MBT",
                        side: "Buy",
                        size: 0,
                        price: 352.80,
                    },
                    {
                        timestamp: "2014-11-06T20:53:00.000Z"
                        symbol: "ACQ",
                        side: "Sell",
                        size: 0,
                        price: 382.90,
                    }
                ],
            },
        }, 
        {
            _id: 'nyse',
            name: 'New York Stock Exchange',
            data: {
                instrument: [
                    {
                        symbol: 'IBM'
                        markPrice: 0,
                    }, 
                    {
                        symbol: 'WMT'
                        markPrice: 0,
                    }
                ],
                orderBooks: [
                    /* Orderbook Data Here */
                ],
                trades: [
                    /* Trades Data Here */
                ],
            },
        }
    ];

I am saving this with something like:

    exchange.websocket_conn.on('message', function (updateData) {
        // Use 'updateData' (a diff) to update exchange.data object.
        // ...

        // Then
        redisClient.hmset(exchange._id.toString(), exchange.data);

        redisClient.publish(exchange._id.toString(), exchange.data);
    });

This works and does publish the data, however I've been reading about 'PSUBSCRIBE' and I'm wondering if this can be broken down a bit further:

I'd like to be able to do something like:

    someOtherRedisClient.subscribe('tsx');
    // Receive All Data from the Exchange Whenever Anything Changes.

    someOtherRedisClient.subscribe('tsx.instrument');
    // Receive 'Instrument' array of All Instruments on Exchange when any Instrument Changes.

    someOtherRedisClient.subscribe(tsx.instrument:MBT');
    // Get Back Only the 'MBT' Instrument Whenever It Changes.

Can the 'Pattern Subscribe' function be used to achieve this?

Thanks,

Answer 1

I'd break down that one big JSON into many JSON's, one for each type of content, e.g.

• Level 1 (e.g. last trade price, best bid/ask)
• Order book
• Trades

and create a seperate topic for each, e.g.

• mktdata:tsx:level1:MBT would have the market price for MBT on the TSX exchange
• mktdata:tsx:orderbook:MBT would be the order book for MBT on the TSX exchange
• mktdata:tsx:trades:MBT could be all the trades but more likely, due to volume, would be better used as a notification to the client to make a seperate query to get the last N trades required from a list

You don't say how many instruments you're writing into Redis, or how many different client applications are consuming the data, but assuming you've not got a huge number of instruments you could indeed use PSUBSCRIBE to get all orderbook updates across the exchange, etc. Given a list of symbols you can also subscribe a long list of channels (e.g. mktdata:tsx:level1:MBT mktdata:tsx:orderbook:MBT mktdata:tsx:level1:ACQ), which can run to tens/hundreds without problem.