CODEWITHSUNDEEP
javascriptnode.jsweb-scrapingcheerio
IscA
IscA

Reputation: 59

Web-scraping images with JavaScript Cheerio

I receive an error when attempting to web-scrape images off amazon. The goal is to download the book cover.

(node:26904) UnhandledPromiseRejectionWarning: Error: undefined is not a valid uri or options object.
    at request (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\request\index.js:44:11)
    at PromisseHandle._RejectOrResolve (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\node-image-downloader\src\image-downloader.js:86:5)
    at new Promise (<anonymous>)
    at ImageDownloader (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\node-image-downloader\src\image-downloader.js:98:26)
    at Request._callback (C:\Users\Isaac\Desktop\webscrape_with_cheerio\index.js:22:13)
    at Request.self.callback (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\request\request.js:185:22)
    at Request.emit (events.js:210:5)
    at Request.<anonymous> (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\request\request.js:1154:10)
    at Request.emit (events.js:210:5)
    at IncomingMessage.<anonymous> (C:\Users\Isaac\Desktop\webscrape_with_cheerio\node_modules\request\request.js:1076:12)
(node:26904) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:26904) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with 
a non-zero exit code.

Here is my code thus far. I believe my error stems from the line;

var imagesrc = $(".a-row center-align litb-on-click img").attr('src')

I need to access the Class over where the book url is, though i might be wrong.

const express = require('express');
const cheerio = require('cheerio');
const download = require('node-image-downloader');
const request = require('request');

const app = express();

app.get('/', (req, res) => {
    var url = "https://www.amazon.com/Black-Swan-Improbable-Robustness-Fragility/dp/081297381X"

    // makeing a request
    request(url,(error, response,html) => {
        if (!error){
            //console.log(html);
            var $ = cheerio.load(html)

            var imagesrc = $(".a-row center-align litb-on-click img").attr('src')

            //download the image

            download({
                imgs: [
                    {
                     uri:imagesrc                       
                    }
                ],
                dest:'./downloads'
            })
                .then((info) => {
                    console.log("Download Complete")
                    process.exit(1)
            })
        }
    })
})
app.listen(5000)

Upvotes: 0

Views: 1251

Answers (1)

Moin Memon
Moin Memon

Reputation: 74

Here, a-row center-align litb-on-click are classes

To select a class we need to use class-selector . (dot)

please try to update your imagesrc variable as below 

var imagesrc = $(".a-row.center-align.litb-on-click img").attr('src');

Spaces are not compulsory.

Upvotes: 0

Related Questions