CODEWITHSUNDEEP
node.jshttploggingcloudmicroservices
Marc Casavant
Marc Casavant

Reputation: 299

Tracking and logging http calls made internally from a node.js server

I'm debugging calls made from my express app to another micro-service on my network. I'm receiving 401 errors and I need to get full raw http logs to give to my security team for analysis.

I'm looking for some advice on tracking HTTP calls from a micro-service I have deployed on Pivotal Cloud Foundry. I've been doing some research and ran across tools like Zipkin and OpenTracing etc.. but those appear to be more about debugging latency and probably do not show HTTP logs. I've also tried using Morgan/Winston modules but they do not track internal calls. Morgan is currently what I'm using to log out the basic HTTP codes but it doesn't pick up on my calls from inside my app either, just the ones made to the app itself from the browser. I need to get the full raw HTTP request to assist the security team. I'm using the default logging output with morgan (STDOUT). I've console logged the headers to see the headers but would like to get them out in a slightly more readable format.

Upvotes: 1

Views: 2422

Answers (1)

shaochuancs
shaochuancs

Reputation: 16226

To log internal HTTP request sent from a Node.js server, you can create a Proxy Node.js server and log all requests there using Morgan.

First, define 3 constants (or read from your project config file):

// The real API endpoint, such as "another micro-service" in your network
const API = http://<real_server>
// Proxy Node.js server running on localhost
const LOGGER_ENDPOINT=http://localhost:3010
// Flag, decide whether logger is enabled.
const ENABLE_LOGGER=true

Second, When your Node.js server is launched, start the logger server at the same time if ENABLE_LOGGER is true. The logger server only do one thing: log the request and forward it to the real API server using request module. You can use Morgan to provide more readable format.

const request = require('request');
const morgan = require('morgan')(':method :url :status Cookie: :req[Cookie] :res[content-length] - :response-time ms');
...
if (ENABLE_LOGGER && LOGGER_ENDPOINT) {
  let loggerPort = 3010;
  const logger = http.createServer((req, res) => {
    morgan(req, res, () => {
      req.pipe(request(API + req.url)).pipe(res);
    });
  });
  logger.listen(loggerPort);
}

Third, in your Node.js server, send API request to logger server when ENABLE_LOGGER is true, and send API directly to the real server when ENABLE_LOGGER is false.

let app = express(); // assume Express is used, but this strategy can be easily applied to other Node.js web framework.
...
let API_Endpoint = ENABLE_LOGGER ? LOGGER_ENDPOINT : API;
app.set('API', API_Endpoint);
...
// When HTTP request is sent internally
request(app.get('API') + '/some-url')...

Upvotes: 1

Related Questions