CODEWITHSUNDEEP
javascriptnode.js
Inn0vative1
Inn0vative1

Reputation: 2145

Node.js: How to determine cause of server hangs

I recently started noticing my Node.js app server hanging after a number of requests. After more investigation I narrowed it down to a specific endpoint. After about 30 hits on this endpoint, the server just stops responding and requests start timing out.

In summary, on each request the clients upload a file, server does some preliminary checks, creates a new processing job, puts that on a queue, using bull, and returns a response to the client. The server then continues to process the job on the queue. The job process involves retrieving job data from a Redis database, clients opening WebSocket connections to check on job status, and server writing to the database once job is complete. I understand any one of those things could be causing the hang up.

There are no errors or exceptions being thrown or logged. All I see is requests start to time out. I am using node-inspector to try to figure out what is causing the hang ups but not sure what to look for.

My question is, is there a way to determine the root cause of hang up, using a debugger or some other means, to figure out too many ABC instances have been created, or too many XYZ connections are open.

Upvotes: 2

Views: 3240

Answers (3)

Adam Carter
Adam Carter

Reputation: 155

Related to OP's issue and solution from my experience:

If you are using the mysql library and get a specific connection connection.getConnection(...) you need to remember to release it afterwards connection.release();

Otherwise your pool will run out of connections and your process will hang.

Upvotes: 0

Inn0vative1
Inn0vative1

Reputation: 2145

In this case it was not actually a Node.js specific issue. I was leaking a database connection in one code path. I learned the hard way, if you exhaust database connections, your Node server will stop responding. Requests just seem to hang.

A good way to track this down, on Postgres at least is by running the query:

SELECT * FROM pg_stat_activity

Which lists all the open database connections, as well as which query was last run on that connection. Then you can check your code for where that query is called, which is super helpful in tracking down leaks.

Upvotes: 4

Matt Fernandez
Matt Fernandez

Reputation: 188

A lot of your node server debugging can be done using the following tools.

Nodemon is an excellent tool to do debugging because it will display errors just like node but it will also attempt to restart your server based on new changes; removing a lot of the stop/start hastle. https://nodemon.io/

Finally I would recommend Postman. Postman lets you manually send in requests to your server, and will help you narrow your search. https://www.getpostman.com/

Upvotes: 1

Related Questions