long poll with node.js and express: how to cache res object into redis or other cache framework

Question

I try to build a chat room webpage with node.js/express/redis on the server side.

Following this snips of code : A Message Wall With Long Poll Properties in Node.JS and Express , I succeed to make one node server running correctly. In this example, the res objects are saved in a list, no any transformation is needed.

But I want to run the node app with pm2 cluster mode(-i 4), so I have to save res object into some place that shared between 4 nodes.

I already used redis in my node project, for the express.session. So I want to cache res into redis.

But the problem occured: when I try to sting-lized res object with JSON.stringify(res), I got :

TypeError: Converting circular structure to JSON

My problem is:

how could I save one res object for later use across node cluster, with redis or something else.

Appreciate.

---

I use util.inspect to print out my container of res (chatroom_id:res) object:

{ '0': null,
  '1390640136999': 
   { domain: null,
     _events: 
      { finish: [Object],
    header: [Function],
    close: [Function: logRequest] },
     _maxListeners: 10,
     output: [],
     outputEncodings: [],
     writable: true,
     _last: false,
     chunkedEncoding: false,
     shouldKeepAlive: false,
     useChunkedEncodingByDefault: false,
     sendDate: true,
     _headerSent: false,
     _header: '',
     _hasBody: true,
     _trailer: '',
     finished: false,
     _hangupClose: false,
     socket: 
      { _connecting: false,
    _handle: [Object],
    _readableState: [Object],
    readable: true,
    domain: null,
    _events: [Object],
    _maxListeners: 10,
    _writableState: [Object],
    writable: true,
    allowHalfOpen: true,
    onend: [Function],
    destroyed: false,
    errorEmitted: false,
    bytesRead: 894,
    _bytesDispatched: 0,
    _pendingData: null,
    _pendingEncoding: '',
    server: [Object],
    _idleTimeout: 120000,
    _idleNext: [Object],
    _idlePrev: [Object],
    _idleStart: 1390640145289,
    parser: [Object],
    ondata: [Function],
    _paused: false,
    _httpMessage: [Circular],
    _peername: [Object] },
     connection: 
      { _connecting: false,
    _handle: [Object],
    _readableState: [Object],
    readable: true,
    domain: null,
    _events: [Object],
    _maxListeners: 10,
    _writableState: [Object],
    writable: true,
    allowHalfOpen: true,
    onend: [Function],
    destroyed: false,
    errorEmitted: false,
    bytesRead: 894,
    _bytesDispatched: 0,
    _pendingData: null,
    _pendingEncoding: '',
    server: [Object],
    _idleTimeout: 120000,
    _idleNext: [Object],
    _idlePrev: [Object],
    _idleStart: 1390640145289,
    parser: [Object],
    ondata: [Function],
    _paused: false,
    _httpMessage: [Circular],
    _peername: [Object] },
     _headers: { 'x-powered-by': 'Express' },
     _headerNames: { 'x-powered-by': 'X-Powered-By' },
     req: 
      { _readableState: [Object],
    readable: true,
    domain: null,
    _events: {},
    _maxListeners: 10,
    socket: [Object],
    connection: [Object],
    httpVersion: '1.0',
    complete: true,
    headers: [Object],
    trailers: {},
    _pendings: [],
    _pendingIndex: 0,
    url: '/robot/chat/query/99/1390640136999/270125/',
    method: 'GET',
    statusCode: null,
    client: [Object],
    _consuming: false,
    _dumped: false,
    httpVersionMajor: 1,
    httpVersionMinor: 0,
    upgrade: false,
    originalUrl: '/robot/chat/query/99/1390640136999/270125/',
    _parsedUrl: [Object],
    query: {},
    res: [Circular],
    next: [Function: next],
    secret: undefined,
    cookies: [Object],
    signedCookies: {},
    sessionStore: [Object],
    sessionID: '4PACUldyCHhT8NgdGY1yz9Pk',
    session: [Object],
    _startTime: Sat Jan 25 2014 16:55:45 GMT+0800 (CST),
    _remoteAddress: '127.0.0.1',
    body: {},
    originalMethod: 'GET',
    _route_index: 2,
    route: [Object],
    params: [Object] },
     locals: [Function: locals],
     end: [Function],
     student_id: '99',
     channel_id: '1390640136999',
     last_msg_id: '270125' } }

There are three [Circular].

---

My pseudocode:

/*
 * query from http client
 * url: /chat/query/:student_id/:channel_id/:last_msg_id/
 */
exports.query = function(req, res){

    // if find some new msg
        // return them as json, immediately
    // else
        // set participator info into res object
        // read the res_list from redis
        // put this res into res_list
        // write back res_list into redis

};


/*
 * notification from other web server: one new msg been created
 * url: /chat/notify/:new_msg_id/
 */
exports.notify = function(req, res){

    // get new_msg from database by id

    // read the res_list from redis
    // for old_res in res_list
        // if this old_res is releated with the new_msg (participator)
            // old_res.sent(json_content)
            // remove this old_res from res_list
    // write back res_list into redis

};

how could i implement those 'read-and-write-back' part?

Answer 1

The short answer: it's not possible to put the res object into a cache and obtain it again from another process, in any meaningful way.

If you think about how long poll works, each HTTP client maintains an open connection to the server, waiting (a long time) for something to be sent back. In other words, when you come to push out new messages, you're just sending data down an already open connection. If you cache your res into redis, what would happen to the connection?

It may not matter though, as long as you have a way to pass the content between your 4 backend processes, they can each update their own set of res connections. New connections would still be load balanced.