CODEWITHSUNDEEP
google-chrome-extensionservice-workerchrome-extension-manifest-v3
byb
byb

Reputation: 355

Chrome extension MV3 with large persistent RAM usage, is it possible?

I am working on a Chrome extension which is designed to inspect the content of any web page the user navigates to, and then alert the user to certain "features" of that content. (Perhaps it is easiest to think of a large number of strings that are searched for in the text data of the web page, though this is a pretty big simplification.) By design, there is a very large set (tens of millions) of features that the extension detects and thus can react to. Each such feature can be represented as a JS number (8 bytes), so the total amount of data might be on the order of 100 MB, or more.

That data can be stored in IndexedDB, but to be able to quickly analyze the page data (as sent by the content script), the background page script (in MV2) or service worker (in MV3) really needs to have the features (which are checked for) stored in RAM, to be able to quickly check a much smaller but still large number of features from the webpage, to see if any of those are present in its own dataset.

This setup actually worked fine in a prototype manifest version 2 (MV2) extension I created. The background script would start by initially grabbing the data from IndexedDB and placing it in a structure in RAM. This takes a bit of time on my laptop (some number of seconds, I don't have precise numbers) but only needs to be done once when the browser is started. After that, the background script is able to quickly react to requests from the content script to check the webpage contents.

Now, trying to make the transition to manifest version 3 (MV3), the problem is that service workers are not persistent, nor even particularly long-lived. So a direct translation would have the service worker do the expensive and slow load from IndexedDB to RAM each time it is restarted. This is clearly not a working setup.

The obvious question then is: Is there some way to avoid Chrome stopping the service worker (and thus have the extension service worker persist for a long period)? If not, is there some way that the data in RAM could stay around, and the service worker grab access to it when starting up? (I am far from an expert on Chrome extensions and service workers, so I apologize if my questions are naive.) I read some discussions that seemed to suggest that perhaps neither of the above is currently possible, but if so, it would basically make the entire concept a non-starter under MV3. Are there any workarounds? (If yes, are those workarounds acceptable in the Chrome Web Store review process?)

I would be very grateful for any pointers!

Upvotes: 0

Views: 1079

Answers (1)

Gardner Bickford
Gardner Bickford

Reputation: 1963

I have encountered a similar situation where the extension is based around a CRDT append-only log which is shared in a p2p fashion. This log will grow too large if duplicate or unnecessary data is added.

First Design

I initially wanted a background script that keeps the log in memory and the content script to notify the background script when it encounters something of interest. This way the background script could compare the new information to what is already in the log and decide wether or not to add/update the info. I looked at the keep-alive workaround and had similar reservations about the Chrome Web Store review process.

MV3 Design

To avoid the in-memory MV2 architecture I have updated the design to use more storage and processing power instead of memory. The content script simply writes anything it would normally send to background script to localStorage. When the service worker wakes up it:

  1. Reads the entire log into memory
  2. Reads all the items which the content script logged
  3. Performs the same logic it would have performed when receiving a message from the content script

The above design does use more storage to store everything the content script encounters. It also has to read the entire log in to memory whenever it does run. It does avoid memory from being wired in 100% of the time.

This architecture reminds me of the tradeoffs made in mobile application development in Android or iOS where, except for in very few circumstances, your task may be archived at any moment. MV3 does need to add something similar to Android's AcquireWakeLock() but I am not holding my breath that will happen any time soon.

Upvotes: 0

Related Questions