CODEWITHSUNDEEP
node.jspostweb-scraping
Austin Leehealey
Austin Leehealey

Reputation: 84

Log into a website from a Nodejs server using a POST request

I am trying to log into a website from my Nodejs server so that I can do some webscraping to gather user data but I am having trouble sending the POST request necessary to log in. On the website it uses a form to receive the user's username and password but I do not know how to send a POST request that perfectly mimics the form.

I have been trying to use request to send the form The first request is there because the form requires an id to be validated and I parse it out of the string so I can send it back in the form.

Whenever I send it it gives me an HTTP error 400 and a blank page that looks like this HTTP ERROR: 400 Problem accessing /pf4/cms2_site/view_deployment. Reason: NoEvent

This is the code I have been using to send the request from my Nodejs server.

        request.get('https://cdm.schoolloop.com/portal/login',{headers: { 'User-Agent': 'Mozilla/5.0' }}, function(err, res, body){

            body = body.substring(body.indexOf('id="form_data_id" value="') + ('id="form_data_id" value="').length, body.length);
            var formDataId = body.substring(0, body.indexOf('"'));
            request.post({
                uri: 'https://cdm.schoolloop.com',
                host: 'https://cdm.schoolloop.com',
                path: '/portal/login?etarget=login_form',
                method: 'POST',
                headers: { 'User-Agent': 'Mozilla/5.0', 'Content-Type': 'application/x-www-form-urlencoded' },
                form: {
                    'login_name': "testUsername",
                    'password': "testPassword",
                    'event.login.x': 0,
                    'event.login.y': 0,
                    'redirect': 'pig4d2db88ad6',
                    'forward': '',
                    'login_form_reverse': '',
                    'form_data_id': formDataId,
                    'sort': '',
                    'reverse': '',
                    'login_form_sort': '',
                    'event_override': 'login',
                    'login_form_filter': '',
                    'login_form_letter': '',
                    'return_url': '',
                    'login_form_page_index': '',
                    'login_form_page_item_count': ''
                }
            }, function (err, firstResponse, body) {
                if (err) {
                    console.log(err);
                } else if (body) {
                    console.log(body);
                }
            });

        });

If anyone could tell me what I am doing wrong in my POST request or how I could figure it out I would appreciate it tremendously.
Thank you in advance!

Upvotes: 1

Views: 1546

Answers (1)

Taymer
Taymer

Reputation: 314

You get the standard response for a POST to 'https://cdm.schoolloop.com'

"request" library identifies the resource throught uri, this should be the correct syntax for the post request:

request.post({
        uri: 'https://cdm.schoolloop.com/portal/login?etarget=login_form',
        method: 'POST',
        headers: { 'User-Agent': 'Mozilla/5.0', 'Content-Type': 'application/x-www-form-urlencoded' },
        form: {
            'login_name': "testUsername",
            'password': "testPassword",
            'event.login.x': 0,
            'event.login.y': 0,
            'redirect': 'pig4d2db88ad6',
            'forward': '',
            'login_form_reverse': '',
            'form_data_id': formDataId,
            'sort': '',
            'reverse': '',
            'login_form_sort': '',
            'event_override': 'login',
            'login_form_filter': '',
            'login_form_letter': '',
            'return_url': '',
            'login_form_page_index': '',
            'login_form_page_item_count': ''
        }
    }, function ...

Upvotes: 1

Related Questions