CODEWITHSUNDEEP
javascriptnode.jsbufferaws-lambdaamazon-kinesis
svnm
svnm

Reputation: 24318

In lambda using node.js how can I parse the data from a kinesis stream

I have the following lambda function set up with kinesis as its event source. The data is coming through as a stream, here I am parsing this to come through as a string e.g. {id=2, sourceIp=220.220.1.220 }. I want to convert this to an object in node.js. How can I do this?

exports.handler = (event, context) => {

    event.Records.forEach(function(record) {
        var data = new Buffer(record.kinesis.data, 'base64').toString('ascii');
        console.log('data: ', data)
    });
};

Upvotes: 2

Views: 3464

Answers (2)

johnnyodonnell
johnnyodonnell

Reputation: 2036

I think what you want to do is this:

var data = JSON.parse(new Buffer(record.kinesis.data, 'base64'));

Upvotes: 3

Squirrel
Squirrel

Reputation: 241

I'm not familiar with the actual output of Kinesis but I'm a bit confused by the example you've given. The test example test even in lambda shows the Kinesis object looking like this

"kinesis": {
    "partitionKey": "partitionKey-3",
    "data": "SGVsbG8sIHRoaXMgaXMgYSB0ZXN0IDEyMy4=",
    "kinesisSchemaVersion": "1.0",
    "sequenceNumber": "49545115243490985018280067714973144582180062593244200961"
}

Which means the parsed string should look something like "id=2,sourceIp=220.220.1.220" from your example. If that is the case you could do something convoluted like this to get the data into a proper object.

var json = {};
data.split(',').map(function(each){
    return each.split('=');
}).forEach(function(pair){
    json[pair[0]] = pair[1];
});

Upvotes: 2

Related Questions