CODEWITHSUNDEEP
curlnode.js
slifty
slifty

Reputation: 13781

cURL equivalent in Node.js?

I'm looking to use information from an HTTP request using Node.js (i.e. call a remote web service and echo the response to the client).

In PHP I would have used cURL to do this. What is the best practice in Node?

Upvotes: 187

Views: 349722

Answers (21)

MisterCat
MisterCat

Reputation: 1631

You have several options for running cURL in javascript/NodeJs,

  1. Using exec command, to execute a command, including cURL
  2. Using node-libcurl, as a binding to the libcurl API
  3. Use the alternative with the same goal! As the goal for cURL is request to HTTP, we have several options here, including:
  • request
  • build in HTTPS
  • got
  • needle
  • etc..

I wrote more about this here: How to use cuRL in Javascript and it's alternative., feel free to read.

Example for exec comamnd

const { exec } = require('child_process');

exec('curl -s https://example.com', (error, stdout, stderr) => {
  if (error) {
    console.error(`Error: ${error}`);
    return;
  }
  if (stderr) {
    console.error(`stderr: ${stderr}`);
    return;
  }
  console.log(`stdout: ${stdout}`);
});

Example for node-libcurl

const { curly } = require('node-libcurl');

async function run() {
  const { statusCode, data, headers } = await curly.get('https://www.google.com')
  
  console.log(statusCode)
  console.log('---')
  console.log(data)
}

run();

Example for Axios (as an alternative)

const axios = require('axios');
const fs = require('fs');

Axios.get('https://example.com')
    .then(response => {
        fs.writeFile('index.html', response.data, (err) => {
            if (err) {
                console.error('Error writing file:', err);
            } else {
                console.log('File saved as index.html');
            }
        });
    })
    .catch(error => {
        console.error('Error fetching URL

:', error); });

I hope it helps!

Upvotes: 0

chladnefazole
chladnefazole

Reputation: 101

Just a thought, if you really want to treat it as cURL request (ie. you just want to check if the page is up or not) you could use (edited from @Glen Thompson answer):

const http = require("http");

function isSiteUp(url) {
  return new Promise((resolve, reject) => {
    http.get(url, (res) => {
      if (!err && res.statusCode == 200) {
        resolve(true);
      } else {
        console.log(err);
        resolve(false);
      }
    })
    .on("error", (err) => {
      reject(err);
    });
  });
}

Upvotes: 0

Alfred
Alfred

Reputation: 61773

How about for example https://github.com/joyent/node/wiki/modules#wiki-tcp. A very quick summary =>

Upvotes: 7

Aral Roca Gomez
Aral Roca Gomez

Reputation: 51

In Node.js 18 you can use directly fetch

const res = await fetch('https://nodejs.org/api/documentation.json');
if (res.ok) {
  const data = await res.json();
  console.log(data);
}

Upvotes: 5

Glen Thompson
Glen Thompson

Reputation: 9996

Here is a standard lib (http) async / await solution.

const http = require("http");

const fetch = async (url) => {
  return new Promise((resolve, reject) => {
    http
      .get(url, (resp) => {
        let data = "";
        resp.on("data", (chunk) => {
          data += chunk;
        });
        resp.on("end", () => {
          resolve(data);
        });
      })
      .on("error", (err) => {
        reject(err);
      });
  });
};

Usage:

await fetch("http://example.com");

Upvotes: 5

Aronis Mariano
Aronis Mariano

Reputation: 2441

EDIT:

For new projects please refrain from using request, since now the project is in maitainance mode, and will eventually be deprecated

https://github.com/request/request/issues/3142

Instead i would recommend Axios, the library is in line with Node latest standards, and there are some available plugins to enhance it, enabling mock server responses, automatic retries and other features.

https://github.com/axios/axios

const axios = require('axios');

// Make a request for a user with a given ID
axios.get('/user?ID=12345')
  .then(function (response) {
    // handle success
    console.log(response);
  })
  .catch(function (error) {
    // handle error
    console.log(error);
  })
  .then(function () {
    // always executed
  });

Or using async / await:

try{
    const response = await axios.get('/user?ID=12345');
    console.log(response)
} catch(axiosErr){
    console.log(axiosErr)
}

I usually use REQUEST, its a simplified but powerful HTTP client for Node.js

https://github.com/request/request

Its on NPM npm install request

Here is a usage sample:

var request = require('request');

request('http://www.google.com', function (error, response, body) {
   if (!error && response.statusCode == 200) {
       console.log(body) // Show the HTML for the Google homepage.
   }
})

Upvotes: 31

D M Patel
D M Patel

Reputation: 86

Use request npm module and after call 

var request = require('request');
request('http://www.google.com', function (error, response, body) {
  console.log('error:', error); // Print the error if one occurred
  console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
  console.log('body:', body); // Print the HTML for the Google homepage.
});

For best practice also use some winston logger module or else simple console.log and then run your application like 

npm start output.txt

Result of above command will generate one txt file on root with all data which you have printed in console.log

Upvotes: 3

Rubin bhandari
Rubin bhandari

Reputation: 1951

You can use request npm module . Super simple to use. Request is designed to be the simplest way possible to make http calls. It supports HTTPS and follows redirects by default.

var request = require('request');
request('http://www.google.com', function (error, response, body) {
  console.log('error:', error); // Print the error if one occurred
  console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
  console.log('body:', body); // Print the HTML for the Google homepage.
});

Upvotes: 1

Indrani Sen
Indrani Sen

Reputation: 407

Request npm module Request node moulde is good to use, it have options settings for get/post request plus it is widely used in production environment too.

Upvotes: 1

Dan Grossman
Dan Grossman

Reputation: 52372

See the documentation for the HTTP module for a full example:

https://nodejs.org/api/http.html#http_http_request_options_callback

Upvotes: 114

Rudolfs
Rudolfs

Reputation: 11

I had a problem sending POST data to cloud DB from IOT RaspberryPi, but after hours I managed to get it straight.

I used command prompt to do so.

sudo curl --URL http://<username>.cloudant.com/<database_name> --user <api_key>:<pass_key> -X POST -H "Content-Type:application/json" --data '{"id":"123","type":"987"}'

Command prompt will show the problems - wrong username/pass; bad request etc.

--URL database/server location (I used simple free Cloudant DB) --user is the authentication part username:pass I entered through API pass -X defines what command to call (PUT,GET,POST,DELETE) -H content type - Cloudant is about document database, where JSON is used --data content itself sorted as JSON

Upvotes: 1

Laksh Goel
Laksh Goel

Reputation: 169

There is npm module to make a curl like request, npm curlrequest.

Step 1: $npm i -S curlrequest

Step 2: In your node file

let curl = require('curlrequest')
let options = {} // url, method, data, timeout,data, etc can be passed as options 
curl.request(options,(err,response)=>{
// err is the error returned  from the api
// response contains the data returned from the api
})

For further reading and understanding, npm curlrequest

Upvotes: 4

Mariano Ruiz
Mariano Ruiz

Reputation: 4749

Uses reqclient, it's a small client module on top of request that allows you to log all the activity with cURL style (optional, for development environments). Also has nice features like URL and parameters parsing, authentication integrations, cache support, etc.

For example, if you create a client object an do a request:

var RequestClient = require("reqclient").RequestClient;
var client = new RequestClient({
        baseUrl:"http://baseurl.com/api/v1.1",
        debugRequest:true, debugResponse:true
    });

var resp = client.post("client/orders", {"client":1234,"ref_id":"A987"}, {headers: {"x-token":"AFF01XX"}})

It will log within the console something like this:

[Requesting client/orders]-> -X POST http://baseurl.com/api/v1.1/client/orders -d '{"client": 1234, "ref_id": "A987"}' -H '{"x-token": "AFF01XX"}' -H Content-Type:application/json
[Response   client/orders]<- Status 200 - {"orderId": 1320934}

The request will return a Promise object, so you have to handle with then and catch what to do with the result.

reqclient is available with npm, you can install the module with: npm install reqclient.

Upvotes: 2

Ravi Teja
Ravi Teja

Reputation: 161

You can try using POSTMAN Chrome app for your request and you can generate node js code from there

Upvotes: 1

jonathancardoso
jonathancardoso

Reputation: 12717

Since looks like node-curl is dead, I've forked it, renamed, and modified to be more curl like and to compile under Windows.

node-libcurl

Usage example:

var Curl = require( 'node-libcurl' ).Curl;

var curl = new Curl();

curl.setOpt( Curl.option.URL, 'www.google.com' );
curl.setOpt( 'FOLLOWLOCATION', true );

curl.on( 'end', function( statusCode, body, headers ) {

    console.info( statusCode );
    console.info( '---' );
    console.info( body.length );
    console.info( '---' );
    console.info( headers );
    console.info( '---' );
    console.info( this.getInfo( Curl.info.TOTAL_TIME ) );

    this.close();
});

curl.on( 'error', function( err, curlErrorCode ) {

    console.error( err.message );
    console.error( '---' );
    console.error( curlErrorCode );

    this.close();

});

curl.perform();

Perform is async, and there is no way to use it synchronous currently (and probably will never have).

It's still in alpha, but this is going to change soon, and help is appreciated.

Now it's possible to use Easy handle directly for sync requests, example:

var Easy = require( 'node-libcurl' ).Easy,
    Curl = require( 'node-libcurl' ).Curl,
    url = process.argv[2] || 'http://www.google.com',
    ret, ch;

ch = new Easy();

ch.setOpt( Curl.option.URL, url );

ch.setOpt( Curl.option.HEADERFUNCTION, function( buf, size, nmemb ) {

    console.log( buf );

    return size * nmemb;
});

ch.setOpt( Curl.option.WRITEFUNCTION, function( buf, size, nmemb ) {

    console.log( arguments );

    return size * nmemb;
});

// this call is sync!
ret = ch.perform();

ch.close();

console.log( ret, ret == Curl.code.CURLE_OK, Easy.strError( ret ) );

Also, the project is stable now!

Upvotes: 35

Gautam Anand
Gautam Anand

Reputation: 19

You might want to try using something like this 

curl = require('node-curl');
curl('www.google.com', function(err) {
  console.info(this.status);
  console.info('-----');
  console.info(this.body);
  console.info('-----');
  console.info(this.info('SIZE_DOWNLOAD'));
});

Upvotes: 0

Nitish Agarwal
Nitish Agarwal

Reputation: 700

you can easily use request module:

https://www.npmjs.com/package/request

Sample code:

var request = require('request');
request('http://www.google.com', function (error, response, body) {
  if (!error && response.statusCode == 200) {
    console.log(body) // Show the HTML for the Google homepage. 
  }
  else {
    console.log("Error "+response.statusCode)
  }
})

Upvotes: 38

strawberryMarshmallow
strawberryMarshmallow

Reputation: 31

I ended up using the grunt-shell library.

Here is my source gist for my fully implemented Grunt task for anyone else thinking about working with the EdgeCast API. You'll find in my example that I use a grunt-shell to execute the curl command which purges the CDN.

This was that I ended up with after spending hours trying to get an HTTP request to work within Node. I was able to get one working in Ruby and Python, but did not meet the requirements of this project.

Upvotes: 1

Jeremy Banks - Vive le Canada
Jeremy Banks - Vive le Canada

Reputation: 129775

The http module that you use to run servers is also used to make remote requests.

Here's the example in their docs:

var http = require("http");

var options = {
  host: 'www.google.com',
  port: 80,
  path: '/upload',
  method: 'POST'
};

var req = http.request(options, function(res) {
  console.log('STATUS: ' + res.statusCode);
  console.log('HEADERS: ' + JSON.stringify(res.headers));
  res.setEncoding('utf8');
  res.on('data', function (chunk) {
    console.log('BODY: ' + chunk);
  });
});

req.on('error', function(e) {
  console.log('problem with request: ' + e.message);
});

// write data to request body
req.write('data\n');
req.write('data\n');
req.end();

Upvotes: 86

warfares
warfares

Reputation: 99

well if you really need a curl equivalent you can try node-curl 

npm install node-curl

you will probably need to add libcurl4-gnutls-dev.

Upvotes: 9

Brad Dickason
Brad Dickason

Reputation: 502

The above examples work but don't go so far as to really deal with a real world example (i.e. when you process data coming in multiple chunks. One thing you need to make sure of is that you have an 'on chunk' handler that push's the data into an array (fastest way to do this in JS) and an 'on end' handler that joins them all together so you can return it.

This is especially necessary when you're working with big requests (5000+ lines) and the server sends a bunch of data at you.

Here's an example in one of my programs (coffeescript): https://gist.github.com/1105888

Upvotes: 8

Related Questions