CODEWITHSUNDEEP
javascriptnode.jsweb-scrapingpuppeteercheerio
Andres Urdaneta
Andres Urdaneta

Reputation: 451

puppeteer web scraping conditionals if statement

Scraping a table... Each country name is within an <a> tag, but some are not. When the structure changes, the program crashes

Code =>

enter image description here

Output =>

enter image description here

I have tried doing the following

const countryName = e.children[1].children[0].children[0].data || 'hello world'

It does not work But also I tried with an IfStatement

const countryName = e.children[1].children[0].children[0].data
if (countryName === undefined) {
   countryName = 'hello world'
}

It did not work either, same output error.

I know what does the error mean... I know that the HTML structure is not the same, but it won't read the conditionals that I'm implementing to give the countryName variable its value

Any ideas?

PD: Same output with cheeriojs

Upvotes: 0

Views: 415

Answers (2)

pguardiario
pguardiario

Reputation: 54984

You probably want something like:

$(e).find('a').first().text() || 'hello world'

You almost never want to resort to using children with cheerio.

Upvotes: 0

vsemozhebuty
vsemozhebuty

Reputation: 13782

You check for undefined too late: any children can be undefined and indexing this undefined with [0] can throw the error.

If your Node.js (V8) or transpiling supports optional chaining and nullish coalescing, you can do this:

const countryName = e?.children?.[1]?.children?.[0]?.children?.[0]?.data ?? 'hello world';

Otherwise, you need this:

const countryName =
  e &&
  e.children &&
  e.children[1] &&
  e.children[1].children &&
  e.children[1].children[0] &&
  e.children[1].children[0].children &&
  e.children[1].children[0].children[0] &&
  e.children[1].children[0].children[0].data ||
  'hello world';

Upvotes: 1

Related Questions