CODEWITHSUNDEEP
node.jsjsontcpmariadb
saifxhatem
saifxhatem

Reputation: 55

Problems sending query results over TCP in a nodejs client/server

I'm trying to make a sort of middleman between my database and other node projects. I realize my code is probably garbage but I'm working on a uni project and have deadlines.

Server code:

const net = require('net');
const port = 7070;
const host = '127.0.0.1';
const mariadb = require('mariadb/callback')

const conn = mariadb.createConnection({
    host: 'localhost',
    port: '3306',
    user: 'root',
    password: '',
    database: 'ecomm'
});

const server = net.createServer();
server.listen(port, host, () => {
    console.log('API server is running on port ' + port + '.');
    //console.log(conn);

});

let sockets = [];
server.on('connection', function(sock) {
    console.log('CONNECTED: ' + sock.remoteAddress + ':' + sock.remotePort);


    sockets.push(sock);

    sock.on('data', function(data) {
        console.log('DATA ' + sock.remoteAddress + ': ' + data);
        if (data == 'fetch') {
            console.log("Fetch operation requested");
            conn.query("SELECT item_id, item_name, item_price, held_by from items", (err, rows) => {
                if (err) console.log("DB connection failed: " + err);
                else {
                    console.log(rows[0].item_price);
                    //sock.write('hehe');
                    var i;
                    for (i = 0; i < 5; i++) {
                        sock.write(JSON.stringify(rows[i]));
                        //sock.write(rows[i]);
                    }

                }
            });
        }
        // Write the data back to all the connected, the client will receive it as data from the server

    });

    // Add a 'close' event handler to this instance of socket
    sock.on('close', function(data) {
        let index = sockets.findIndex(function(o) {
            return o.remoteAddress === sock.remoteAddress && o.remotePort === sock.remotePort;
        })
        if (index !== -1) sockets.splice(index, 1);
        console.log('CLOSED: ' + sock.remoteAddress + ' ' + sock.remotePort);
    });
});

process.on('uncaughtException', function (err) {
    console.error(err);
    //console.log("Client disconnected");
  });

Client code:

const net = require('net');
const client = new net.Socket();
const port = 7070;
const host = '127.0.0.1';


client.connect(port, host, function() {
    console.log('API connection successful.');
    client.write('fetch');

});
client.on('data', function(data) {
    //console.log('Server Says : \n' + JSON.parse(data));
    var parsed_data = JSON.parse(data);

    console.log(parsed_data);
});

client.on('close', function() {
    console.log('Connection closed');
});

Now the server, upon getting a fetch request, will get all items from the database, and then throw it back to the client. The problem is, if I send 1 or 2 items; it works fine. If I send more than that, I get a parsing error and the client crashes.

    SyntaxError: Unexpected token { in JSON at position 64
    at JSON.parse (<anonymous>)
    at Socket.<anonymous> (C:\Users\saif\Desktop\CliServTest\client.js:14:2
    at Socket.emit (events.js:198:13)
    at addChunk (_stream_readable.js:288:12)
    at readableAddChunk (_stream_readable.js:269:11)
    at Socket.Readable.push (_stream_readable.js:224:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)

Upvotes: 0

Views: 521

Answers (1)

jfriend00
jfriend00

Reputation: 707686

This code here in the client:

client.on('data', function(data) {
    //console.log('Server Says : \n' + JSON.parse(data));
    var parsed_data = JSON.parse(data);

    console.log(parsed_data);
});

is a problem. TCP is a stream protocol and there are no guarantees what chunks your data arrives in. At some point in the size of data you send (which may even depend a bit on the local infrastructure), your data will get broken into multiple chunks and will arrive to the client.on('data', ...) in pieces in separate data events. The JSON format cannot be parsed in pieces. You need to have a whole piece of JSON data before you can parse it.

So, you're likely getting a partial piece of JSON and trying to parse it.

This is typically one major reason why people use higher level protocols or higher level libraries because they can combine all your chunks of data back into the original piece that was sent and then give you a whole piece. For example, webSocket will do that for you.

So, you have to write that that is capable of accumulating data from multiple incoming data events, combine them together, figure out where the appropriate JSON parsing boundary is and then call JSON.parse() with a whole piece of JSON.

This means either putting your own markers into the TCP stream that you can look for as a delimiter, sending a length or using some higher level protocol (like webSocket) that does this for you.

Here's are some related answers that discuss some of the options: Detect complete data received on 'data' Event in Net module of Node.js and How do I effectively send a large packet / combine smaller packets?.

If you were locking for the quickest to code solution, I'd probably just switch to using webSocket instead of plain TCP since a webSocket will deliver you the same packet of data that was sent all at once. It delimits the data for you, accumulates incoming pieces, recombines them into the same chunk that was originally sent and then only notifies you when a whole chunk has arrived.

Upvotes: 1

Related Questions