CODEWITHSUNDEEP
web-scrapinggoogle-chrome-extension
Hugues
Hugues

Reputation: 653

Scraping web site using Google Chrome extension

I am trying to build a chrome extension to be used by many users. This chrome extension will always scrape data from the same web site.

Following on-line trainings, I understand that I have to place the scraping logic in a content script. Now as the web site to be scraped contains many pages and many links, I am trying to do this in a way where the user of the extension does not see the main window opening different links.

You'll find below the starting point of the content script

chrome.runtime.onMessage.addListener(function(request,sender, sendResponse){
  if (request.todo=="extractData") {
  alert("before launching the request");
  const request = require('request');
  request('https://www.url_to_scrape.com', function(err, res, body) {
      alert("in the request");
      console.log(body);
  });
  }
});

I am getting the following error message : "Unchecked runtime.lastError: The message port closed before a response was received."

Any help would be very much appreciated :-)

Hugues

Upvotes: 0

Views: 517

Answers (1)

Hugues
Hugues

Reputation: 653

Found the answer, the code should be inserted in the background page, not in the content page.

Also, the list of websites to be crawled should be added in the manifest.json

Upvotes: 1

Related Questions