CODEWITHSUNDEEP
google-chromeweb-scrapinggetpostmangoogle-chrome-devtools
lu5er
lu5er

Reputation: 3564

Postman interceptor request running forever

I am trying to intercept a website - https://www.kroger.com/pl/chicken/05002. In the chrome network tab, I see the request as below, with the details of the products nicely listed as JSON

enter image description here

I copied the cURL as bash and imported it as raw text in Postman. It ran forever without any response. Then I used the intercept feature and still it is running forever.

enter image description here

When both the requests are exactly same, why is it running in Chrome and not in Postman? What am i missing? Any help is appreciated, thanks in advance.

Upvotes: 1

Views: 322

Answers (1)

Stroh
Stroh

Reputation: 45

This is probably happening because they don't want you to do what you are trying to do. Note the "filter.verified" param in the URL.

You may want to try reaching out to them for an external API token - especially if you are creating an app or extension to compare competitive prices with the intention of distributing said app or extension - regardless of if it is for financial compensation or not.

Ethically questionable workaround (which would defintely need to be improved upon - this is simply an example of how you could solve your problem...):

GET https://www.kroger.com/search?query=chicken&searchType=default_search&fulfillment=all

const html = cheerio(responseBody);

var results = [];

html.find('div[class="AutoGrid-cell min-w-0"] > div').each(function (i, e)
{
    results.push({
        "Item": e.children[e.children.length-3].children[0].children[0].children[0]["data"],
        "Price": e.children[e.children.length-4].children[0].attribs["value"]
    })
});

console.log(results);

Output

If you are unable to obtain an API token from them, this would probably be a legal way to accomplish what you want.

Upvotes: 1

Related Questions