CODEWITHSUNDEEP
javascriptnode.jsweb-scrapingaxiosscreen-scraping
ProMaker Dev
ProMaker Dev

Reputation: 155

shorten axios text/html response

I am sending an Axios request to a web page for scraping a little string off of it, but the returned response is a large html, and I only need a little part of it, is there a way to somehow shorten the response so I can save data and make the request faster?

const longHtml = await axios.get('https://example.com');
const shortHtml = longHtml.data //get short data here

Upvotes: 0

Views: 2313

Answers (2)

Harry Kruger
Harry Kruger

Reputation: 317

If I understand what your trying to do, you wan't to stop the request when you find the data you want, you could use htmlparser2 and feed it a stream from axios and then register listeners and when you get the element you need you can end the stream.

Upvotes: 1

Kidas
Kidas

Reputation: 329

Web scraping is a technique used for retrieving data from websites. You fetch the page's contents, and after that extract the data you need from the page.

here an example by using axios and cheerio

const axios = require("axios")
const cheerio = require("cheerio")

async function fetchHTML(url) {
  const { data } = await axios.get(url)
  return cheerio.load(data)
}
const $ = await fetchHTML("https://example.com")

// Print the full HTML
console.log(`Site HTML: ${$.html()}\n\n`)

// Print some specific page content
console.log(`First h1 tag: ${$('h1').text()}`)

Upvotes: 1

Related Questions