CODEWITHSUNDEEP
google-chrome-extension
Brad Parks
Brad Parks

Reputation: 72001

Chrome Extension - Scrape any url, ignoring sandboxing and Content Security Policy?

I'd like to build a chrome extension that can make requests against any web page that the user has access to, even pages that are protected by Content Security Policies, preferably in the background (without having to have the page open in the browser).

So for example, I'd like to be able to:

Is this possible? It seems like I could have the extension open a new window, and a tab for every page I want to pull info from. Is this the only way this can work? I'd prefer to have this happen behind the scenes, without having to open a window.

Upvotes: 0

Views: 137

Answers (1)

woxxom
woxxom

Reputation: 73556

CSP is not a problem as long as your manifest.json adds the URLs you want to process in permissions e.g. "*://*/" or "<all_urls>" will allow access to any site.

The solution, however, depends on how that page is built. If the server response contains all the info you need then you can simply make a direct request via XMLHttpRequest or fetch (more info) in the background script, parse it with DOMParser and extract the data. Otherwise you can try to run it in an iframe (you'll have to strip X-Frame-Options) or in an inactive/pinned tab and use a content script to extract the data. To access JavaScript variables of the page you'll need to add a DOM script so its code will run in page context.

Upvotes: 1

Related Questions