CODEWITHSUNDEEP
javascriptloopsobjectpuppeteer
freemanfl
freemanfl

Reputation: 53

JS iterating through object

I'm trying to create a puppeteer scraper. It has do go through urls that are stored in an object and execute the function like this:

    (async () => {const url = 'https://majestic.com/';
        const browser = await puppeteer.launch({headless: true});
        const page = await browser.newPage();
        await page.setCookie(... cookies);
        await page.goto('https://majestic.com/reports/site-explorer?q='+ newDataDomains[2]['Domain'] + '&oq=' + newDataDomains[2]['Domain'] + '&IndexDataSource=F');
    
        const rootDomainTF = await page.evaluate(() => (document.querySelector("#trust_flow_chart > svg > g > g > text").textContent));
        console.log('Root Domain TF = ' + rootDomainTF);
};

It goes to a url constructed via url from a domain (currently just a single one as example) and gets the value of some html.

Object newDataDomains has around 100 elements similar to

[
  { Domain: 'surfury.co.uk', 'Number of drops': 1 },
  { Domain: 'tal0ne.co.uk', 'Number of drops': 1 },
  { Domain: 'devonsfinest.co.uk', 'Number of drops': 2 }
]

Number of drops variable is also important, on each iteration fucntion will be slightly different depenging of the value of Number of drops

I have written the code for what should get scraped, but now I need a way to tell the program "go through this object's urls and also keep the value of both properties. Any help with how to implement this? Thanks in advance

Upvotes: 1

Views: 82

Answers (1)

Mpendulo Philip Sibiya
Mpendulo Philip Sibiya

Reputation: 84

Use a For Loop to iterate through the Object and declare two arrays one for the URL and the other for the Number of drops (that's how you'll go through the object and store its values). and to access Object values use the dot method Example: objectName.url.

Upvotes: 1

Related Questions