CODEWITHSUNDEEP
javascriptnode.jsexpress
matpen
matpen

Reputation: 291

Express.js - abort request on timeout

I am exploring ways to abort client requests that are taking too long, thereby consuming server resources. Having read some sources (see below), I tried a solution like the following (as suggested here):

const express = require('express');
const server = express();
server
    .use((req, res, next) => {
        req.setTimeout(5000, () => {
            console.log('req timeout!');
            res.status(400).send('Request timeout');
        });
        res.setTimeout(5000, () => {
            console.log('res timeout!');
            res.status(400).send('Request timeout');
        });
        next();
    })
    .use(...) // more stuff here, of course
    .listen(3000);

However, it seems not to work: the callbacks are never called, and the request is not interrupted. Yet, according to recent posts, it should.

Apart from setting the timeout globally (i.e. server.setTimeout(...)), which would not suit my use case, I have seen many suggesting the connect-timeout middleware. However, I read in its docs that

While the library will emit a ‘timeout’ event when requests exceed the given timeout, node will continue processing the slow request until it terminates.
Slow requests will continue to use CPU and memory, even if you are returning a HTTP response in the timeout callback.
For better control over CPU/memory, you may need to find the events that are taking a long time (3rd party HTTP requests, disk I/O, database calls)
and find a way to cancel them, and/or close the attached sockets.

It is not clear to me how to "find the events that are taking long time" and "a way to cancel them", so I was wondering if someone could share their suggestions. Is this even the right way to go, or is there a more modern, "standard" approach?

Specs:

Sources:

Edit: I have seen some answers and comments offering a way to setup a timeout on responses or "request handlers": in other words, the time taken by the middleware is measured, and aborted if it takes too long. However, I was seeking for a way to timeout requests, for example in the case of a client sending a large file over a slow connection. This happens probably even before the first handler in the express router, and that is why I suspect that there must be some kind of setting at the server level.

Upvotes: 2

Views: 2387

Answers (1)

Mike
Mike

Reputation: 1375

Before rejecting long request, I think, it's better to measure requests, find long ones, and optimize them. if it is possible.

How to measure requests?

Simplest way it is to measure time from start, till end of request. You'll get Request Time Taken = time in nodejs event loop + time in your nodejs code + wait for setTimeout time + wait for remote http/db/etc services

If you don't have many setTimeout's in code, then Request Time Taken is a good metric. (But in high load situations, it becomes unreliable, it is greatly affected by time in event loop )

So you can try this measure and log solution http://www.sheshbabu.com/posts/measuring-response-times-of-express-route-handlers/

How to abort long request

it all depends on your request handler

Handler does Heavy computing

In case of heavy computing, which block main thread, there's noting you can do without rewriting handler.

if you set req.setTimeout(5000, ...) - it fires after res.send(), when main loop will be unlocked

 function (req, res) {
  for (let i = 0; i < 1000000000; i++) {
    //blocking main thread loop
  }
  res.send("halted " + timeTakenMs(req));
}

So you can make your code async, by injecting setTimeout(, 0) some where; or move computing to worker thread

Handler has many remote requests

I simulate remote requests with Promisfied setTimeout

async function (req, res) {
    let delayMs = 500;
    await delay(delayMs); // maybe axios http call
    await delay(delayMs); // maybe slow db query
    await delay(delayMs);
    await delay(delayMs);
    res.send("long delayed" + timeTakenMs(req));
  }

In this case you can inject some helpers to abort your request chain blockLongRequest - throws error if request time is too big;

async function (req, res) {
    let delayMs = 500;
    await delay(delayMs); // mayby axios call
    blockLongRequest(req);
    await delay(delayMs); // maybe db slow query
    blockLongRequest(req);
    await delay(delayMs);
    blockLongRequest(req);
    await delay(delayMs);
    res.send("long delayed" + timeTakenMs(req));
  })

single remote request

function (req, res) {
    let delayMs = 1000;
    await delay(delayMs);
    //blockLongRequest(req); 
    res.send("delayed " + timeTakenMks(req));
}

we don't use blockLongRequest because it's better to deliver answer instead of error. Error may trigger client to retry, and you get your slow requests doubled.

Full example

(sorry for TypeScript, yarn ts-node sever.ts )

import express, { Request, Response, NextFunction } from "express";

declare global {
  namespace Express {
    export interface Request {
      start?: bigint;
    }
  }
}

const server = express();
server.use((req, res, next) => {
  req["start"] = process.hrtime.bigint();
  next();
});
server.use((err: any, req: Request, res: Response, next: NextFunction) => {
  console.error("Error captured:", err.stack);
  res.status(500).send(err.message);
});

server.get("/", function (req, res) {
  res.send("pong " + timeTakenMks(req));
});
server.get("/halt", function (req, res) {
  for (let i = 0; i < 1000000000; i++) {
    //halting loop
  }
  res.send("halted " + timeTakenMks(req));
});
server.get(
  "/delay",
  expressAsyncHandler(async function (req, res) {
    let delayMs = 1000;
    await delay(delayMs);
    blockLongRequest(req); //actually no need for it
    res.send("delayed " + timeTakenMks(req));
  })
);
server.get(
  "/long_delay",
  expressAsyncHandler(async function (req, res) {
    let delayMs = 500;
    await delay(delayMs); // mayby axios call
    blockLongRequest(req);
    await delay(delayMs); // maybe db slow query
    blockLongRequest(req);
    await delay(delayMs);
    blockLongRequest(req);
    await delay(delayMs);
    res.send("long delayed" + timeTakenMks(req));
  })
);

server.listen(3000, () => {
  console.log("Ready on 3000");
});


function delay(delayTs: number): Promise<void> {
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve();
    }, delayTs);
  });
}
function timeTakenMks(req: Request) {
  if (!req.start) {
    return 0;
  }
  const now = process.hrtime.bigint();
  const taken = now - req.start;

  return taken / BigInt(1000);
}
function blockLongRequest(req: Request) {
  const timeTaken = timeTakenMks(req);
  if (timeTaken > 1000000) {
    throw Error("slow request - aborting after " + timeTaken + " mks");
  }
}

function expressAsyncHandler(
  fn: express.RequestHandler
): express.RequestHandler {
  return function asyncUtilWrap(...args) {
    const fnReturn = fn(...args);
    const next = args[args.length - 1] as any;
    return Promise.resolve(fnReturn).catch(next);
  };
}

I hope, this approach helps you to find an acceptable solution

Upvotes: 1

Related Questions