CODEWITHSUNDEEP
node.jsweb-scraping
Frederik 84
Frederik 84

Reputation: 61

Web scraping Get responsetext to server

Hi guys Ive been struggling for a couple of weeks trying to get this to work. Ive read a ton and im almost close to an solution. I was just hoping someone with more experience then me could help out.

At this point I have:

To test my server I have used this code:

var http = require('http');
http.createServer(function (req, res) {
 res.writeHead(200, {'Content-Type': 'text/plain'});
 res.end('Hello World test sucess\n');
}).listen(8080); 
console.log('Server is running at http://178.62.253.206:8080/');

Which is working fine

I just manged to get my scraper script to get the html response text loaded into console using this: 

var request = require('request');
var cheerio = require('cheerio');

request('http://www.xscores.com/soccer', function (error, response, html) {
  if (!error && response.statusCode == 200) {
    console.log(html);
  }
});

I really want to somehow merge these 2 codes. Meaning I would like to load the response text into my server. I've tried a couple of things but I'm not exactly sure how I should phrase the code.

Any help is much appreciated

frederik

Upvotes: 0

Views: 49

Answers (2)

kgangadhar
kgangadhar

Reputation: 5088

Since you are sending html content, change the content-type to 'text/html' and try this:

var http = require('http');
var request = require('request');
var cheerio = require('cheerio');

http.createServer(function (req, res) {

    request('http://www.xscores.com/soccer', function (error, response, html) {
        if (!error && response.statusCode == 200) {
            console.log(html);
            res.writeHead(200, {
                'Content-Type': 'text/html'
            });
            res.end(html);
        }
    });
}).listen(8080);


console.log('Server is running at http://178.62.253.206:8080/');

Upvotes: 1

Terry Lennox
Terry Lennox

Reputation: 30675

It should look something like this!

var http = require('http');
var request = require('request');
var cheerio = require('cheerio');

http.createServer(function (req, res) {
   request('http://www.xscores.com/soccer', function   (error, response, html) {
   if (!error && response.statusCode == 200) {
       res.writeHead(200, { 'Content-Type':'text/plain'});
       res.end('html:'+html);
   }

}); }).listen(8080); console.log('Server is running at http://178.62.253.206:8080/');

Upvotes: 0

Related Questions