How to create object using javascript Map() function

Question

I'm learning Map() function of javascript. I want to map rules for declarativeNetRequest but I need a bit of help.

How I can create a map using this model as output:

{
    "id" : 1,
    "priority": 1,
    "action" : { "type" : "block" },
    "condition" : {
        "urlFilter" : "abc",
        "domains" : ["foo.com"],
        "resourceTypes" : ["script"]
  }
}

At the moment I'm fetching a list of filters from a remote server and I'm parsing it using this code, after the parsing I want to map the results to have an object that I can use with declarativeNetrequest api

let def = {};
let out;
let defMap = new Map();

const fetch = async () => {
  await axios({
  url: "https://raw.githubusercontent.com/easylist/easylist/master/easylist/easylist_adservers.txt",
  responseType: "text"
  })
  .then( (response) => {
      let parsed = response.data.split("\n");
    out = parsed.filter( (item) => {
      if( !item.startsWith("!") ){
        return item.replace(/([\|\|\w\d\.\^]+)(?:[\w-\?=~,\$]+)$.*/, "$1");
      }
    });
  });
  return out;
}
// debug only 
fetch().then( (out) => {
  console.log(out);
});

will be this possible?

Answer 1

See below example that outputs a list of JSON objects based on the rules. Here are some other pointers:

• You don't need a Map(), just a regular JSON/Javascript object structure
• You don't need to use .then() (Promises syntax) and await (async/await syntax) together - just chose one - await is the more modern way to deal with asynchronous responses - I used await
• If you had used .then(), then the return statement would have completed before the inner callback function had been called, so out would not have been set
• I didn't look into the syntax of the file too much - but I presume fragments after ^ are useful, perhaps applying them to only certain subdomains
• The block rule accepts an array of domains - so you may be able to create one rule that applies to many domains instead of one rule for each domain

import axios from 'axios'

async function fetch() {
  const resp = await axios({
    url: "https://raw.githubusercontent.com/easylist/easylist/master/easylist/easylist_adservers.txt",
    responseType: "text"
  })
  return [...parseDomains(resp.data)].map(blockRule)
}

function* parseDomains(data) {
  for (const line of data.split("\n")) {
    const match = line.match(/^[\|]+(?<domain>[^\^]+)/)
    if (match) yield match.groups.domain
  }
}

function blockRule(domain, id) {
  return {
    id,
    priority: 1,
    action: { type: "block" },
    condition: {
      urlFilter: `for_${domain}`,
      domains: [domain],
      resourceTypes: ["script"]
    }
  }
}

// debug only 
fetch().then((out) => {
  console.log(out)
})

Additional explanation:

1. Get the data:

resp = await axios(...)

2. parseDomains splits the data into lines (as you did before), and does a regular expression match on it. If it matches, it'll yield another domain. The function* makes this a generator expression - basically it can be iterated over (like an array) the elements are whatever you yield. If each line would yield a domain then you could have used .map() to convert each line to a domain, but only some lines yield domains so you need something else. An alternative would have been to create an empty array and then add a new domain element to it each time you found a valid domain.

The regular expression itself is a bit simpler than the one you had:

• match one or more |'s
• followed by a one or more non-^ (so will end/terminate when ^ or end-of-string is encountered) - this is put into a match group (enclosed by parentheses, like you did) and given the name domain
• if the return value, match is truthy, it means you have found a match, and the matches can be found in match.groups - since I named it domain it's in match.groups.domain

3. Since parseDomains is a generator expression, which are lazily evaluated, to run a map/filter on it you'll need to eagerly evaluate it - that's what [...expr] does - it basically forces the generator expression to be evaluated and converted into an array. We could have kept evaluating things lazily like this instead:

async function* fetch() {
  const resp = await axios(...)
  for (const domain of parseDomains(resp.data)
    yield blockRule(domain)
}

4. Finally we map the domains through blockRule to output the actual rules, so you'll have an array of rule objects in the end.