CODEWITHSUNDEEP
javascriptnode.jsweb-scrapingcheerio
heyitsvajid
heyitsvajid

Reputation: 1121

Scraping all rows from table using cheerio

I'm trying to scrape all rows from the Price Performance table on webpage https://www.barchart.com/stocks/quotes/aapl/performance It is the table below the line chart. The table doesn't have any id or class attribute. I'm trying to get through the table from the div it is included in. Below is the code but it is not printing the text inside td elements.

$ = await fetchData(performanceHistoryUrl);    
let performanceTableDiv = $(".bc-table-scrollable-inner") // Class of div which is enclosing table
    var childSelector = 'table' // table selector 
    var performanceTable = performanceTableDiv.find(childSelector)
    performanceTable.each((index, element) => {
            if (index === 0) return true;
            const tds = $(element).find("td");
            const colOne = $(tds[0]).text();
            const colTwo = $(tds[1]).text();
            const colThree = $(tds[2]).text();
            const tableRow = { colOne, colTwo, tableRow };
            console.log(tableRow);
    });

Upvotes: 2

Views: 4202

Answers (1)

ggorlen
ggorlen

Reputation: 57175

This seems to give the data you want:

const axios = require("axios");
const cheerio = require("cheerio");

axios.get("https://www.barchart.com/stocks/quotes/aapl/performance")
  .then(({data}) => {
    const $ = cheerio.load(data);
    const rows = [];
    const sel = ".bc-symbol-performance-widget:nth-child(1) table tr";
    $(sel).each(function (i, e) {
      const row = [];
      rows.push(row);
      $(this).find("th, td").each(function (i, e) {
        row.push($(this).text().trim());
      });
    });
    console.table(rows);
  });

Output:

┌─────────┬───────────┬─────────────────┬─────────────────────┐
│ (index) │     0     │        1        │          2          │
├─────────┼───────────┼─────────────────┼─────────────────────┤
│    0    │ 'Period'  │ 'Made New high' │ 'Percent From Last' │
│    1    │  '5-Day'  │   '2  times'    │      '-5.27%'       │
│    2    │ '1-Month' │   '8  times'    │      '-5.27%'       │
│    3    │ '3-Month' │   '24  times'   │      '-5.27%'       │
│    4    │ '6-Month' │   '26  times'   │      '-5.27%'       │
│    5    │   'YTD'   │   '14  times'   │      '-5.27%'       │
│    6    │ '52-Week' │   '25  times'   │      '-5.27%'       │
│    7    │ '2-Year'  │   '91  times'   │      '-5.27%'       │
│    8    │ '3-Year'  │   '40  times'   │      '-2.03%'       │
│    9    │ '5-Year'  │   '76  times'   │      '-2.03%'       │
│   10    │ '10-Year' │   '45  times'   │      '-2.03%'       │
│   11    │ '20-Year' │   '88  times'   │      '-2.03%'       │
└─────────┴───────────┴─────────────────┴─────────────────────┘

This gives the first table, New Highs. If you want the New Lows table, you can try the selector ".bc-symbol-performance-widget:nth-child(3) table tr".

I used Axios because I had it handy, but node-fetch works just the same (I assume Cheerio is the issue, not the request).

Revisiting this a year later, the selectors and table layout have changed. Here's new code that works with the update (the third column is empty so I remove it with not):

import cheerio from "cheerio"; // 1.0.0-rc.12

fetch("https://www.barchart.com/stocks/quotes/aapl/performance")
  .then(res => res.text())
  .then(html => {
    const rowSelector = "barchart-table-scroll table tr";
    const colSelector = "th:not(:nth-child(3)), td:not(:nth-child(3))";
    const $ = cheerio.load(html);
    const rows = [...$(rowSelector)].map(e =>
      [...$(e).find(colSelector)].map(e => $(e).text().trim())
    );
    console.table(rows);
  });

Although I'm using slightly different syntax, the pattern is the same: map over the rows, then for each row, make an array of its cells.

Upvotes: 8

Related Questions