CODEWITHSUNDEEP
javascriptnode.jscookies
MrD
MrD

Reputation: 2481

NodeJS - How to get cookies from server response

I want to use nodeJS as tool for website scrapping. I have already implemented a script which logs me in on the system and parse some data from the page.

The steps are defined like:

  1. Open login page

  2. Enter login data

  3. Submit login form

  4. Go to desired page

  5. Grab and parse values from the page

  6. Save data to file

  7. Exit

Obviously, the problem is that every time my script has to login, and I want to eliminate that. I want to implement some kind of cookie management system, where I can save cookies to .txt file, and then during next request I can load cookies from file and send it in request headers.

This kind of cookie management system is not hard to implement, but the problem is how to access cookies in nodejs? The only way I found it is using request response object, where you can use something like this: 

 request.get({headers:requestHeaders,uri: user.getLoginUrl(),followRedirect: true,jar:jar,maxRedirects: 10,},function(err, res, body) {
        if(err) {
            console.log('GET request failed here is error');
            console.log(res);
        }

        //Get cookies from response
        var responseCookies = res.headers['set-cookie'];
        var requestCookies='';
        for(var i=0; i<responseCookies.length; i++){
            var oneCookie = responseCookies[i];
            oneCookie = oneCookie.split(';');
            requestCookies= requestCookies + oneCookie[0]+';';
        }
    }
);

Now content of variable requestCookies can be saved to the .txt file and can loaded next time when script is executed, and this way you can avoid process of logging in user every time when script is executed.

Is this the right way, or there is a method which returns cookies?

NOTE: If you want to setup your request object to automatically resend received cookies on every subsequent request, use the following line during object creation: 

var request = require("request");
request = request.defaults({jar: true});//Send cookies on every subsequent requests

Upvotes: 8

Views: 31294

Answers (3)

Italo Borssatto
Italo Borssatto

Reputation: 15689

This function gets a specific cookie value from a server response (in Typescript):

function getResponseCookieValue(res: Response, param: string) {
    const setCookieHeader = res.headers.get('Set-Cookie');
    const parts = setCookieHeader?.match(new RegExp(`(^|, )${param}=([^;]+); `));
    const value = parts ? parts[2] : undefined;
    return value;
}

Upvotes: 1

kromatix
kromatix

Reputation: 17

I use Axios personally.

  axios.request(options).then(function (response) {
  console.log(response.config.headers.Cookie)
}).catch(function (error) {
      console.error(error)
});

Upvotes: -2

Saeger
Saeger

Reputation: 324

In my case, i've used 'http'library like the following:

http.get(url, function(response) {
    variable = response.headers['set-cookie'];
})

Upvotes: 10

Related Questions