Node: Check if URL is safe to request

Question

I am collecting URLs from visitors of my site that I then (on-demand) request from my backend. However, I have gotten stuck on the validation of the URL.

A URL is safe to request when:

• It is a valid URL
• It is unambiguous (in RFC 1918 terms, it is public)

What I have found so far:

• The URL interface (validates URLs)
• The ipaddr.js library (validates IPs)
• The is-valid-domain library (Looks to be broken, since it returns true for 256.256.256.256)

However, none of these seems to do what I need it to - not even a combination of them does.

Is there anything I have overlooked?

Answer 1

I solved my own problem. I used:

• The URL interface for URL validation
• The hostname-is-private NPM module

First, I validate the input with new URL(input), and then I use require('hostname-is-private').isPrivate (this is an async function) on that URL.hostname. This also works on IP addresses (even if the name does not suggest that).

The code looks something like this:

let isPrivate = require('hostname-is-private').isPrivate;

function validateURL(urlstr, cb) {
  try {
    let url = new URL(urlstr);
    isPrivate(url.hostname, (err, res) => {
      if (err) {cb(false)}
      cb(!res);
    })
  } catch (e) {
    return cb(false);
  }
}