Website and extension using same model

[escottgoodwin]

Per the chrome extension example, you pack the model with the extension. Is there a way for a website and chrome extension to use the same cached model? If my project has both a website and extension, I hope they could use a single model instead of having store 2 on the user's machine.

[xenova]

Could you maybe explain a little more about your intended use-case (and why you need the model to run on both a website and a chrome extension)? If there's a way to access the cache of a specific website from your chrome extension (assuming permissions are set correctly), there might be a sneaky way to fix your problem. Basically, you would only load (and cache) the model under your website domain, and then access this cached model from the chrome extension. However, I don't think you'll be able to do this if you aren't actively visiting the page (but feel free to try yourself).

Usually, the answer to "shared models" across domains/extensions is that it isn't possible due security protocols put in place by browsers to restrict cross-domain sharing of cached resources. Allowing one website to read the cache of another can most certainly be abused, so, this is a hard limitation for sharing cached models across websites.

[xenova]

I've been working on a small proof-of-concept extension where the model is downloaded separately: https://github.com/xenova/transformers.js/tree/extension/examples/extension

Let me know if that solves your problem!

[escottgoodwin]

Thanks for the reply. So it seems you are putting the model in chrome local storage for the cache. Isn't local storage reserved by domain and/or extension. My use case is would be a website that has an integrate extension (same login, data access). You can use a transformer model to process text while on the website. But you can use the same model to process text while browsing the web and save the results to your logged in account. It seems a this would require two copies of the same model being cached on the user's computer. An ideal workflow would be: User signs up User downloads/caches transformer.js model to process text on the website. User install extension User access the already downloaded model to do the same type of text processing while browsing the web. I am not sure this is possible because of the security concerns. I didn't know if there was some permission setting in the extension that could allow access to a already cached model. It's just that having two copies of the same 1GB+ model on the user's machine is less than ideal.

…

On Sun, Jul 16, 2023 at 8:57 AM Joshua Lochner ***@***.***> wrote: I've been working on a small proof-of-concept extension where the model is downloaded separately: https://github.com/xenova/transformers.js/tree/extension/examples/extension Let me know if that solves your problem! — Reply to this email directly, view it on GitHub <#182 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFZCSEOM7HZ32B7SE67TSJTXQQFOTANCNFSM6AAAAAA2AYH5EQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[xenova]

Isn't local storage reserved by domain and/or extension.

The naming is a bit confusing (since the localStorage property is - as you said - reserved by domain), but the extensions API calls this the Storage API which is divided into four buckets ("storage areas"), one of which is storage.local. So my proof-of-concept is able to use the same model across different domains (or even across extensions; see here).

But you can use the same model to process text while browsing the web and save the results to your logged in account.

In this case, you could look into the externally_connectable option (https://developer.chrome.com/docs/extensions/mv3/messaging/#external-webpage), which allows you to receive and respond to messages from your website. For more info, see this twitter thread.

It seems a this would require two copies of the same model being cached on the user's computer.

Combining my proof-of-concept with externally_connectable should mean you only need one copy (which is stored/managed by the extension).

[escottgoodwin]

Ok, this sounds great.

…

On Sun, Jul 16, 2023 at 10:10 AM Joshua Lochner ***@***.***> wrote: Isn't local storage reserved by domain and/or extension. The naming is a bit confusing (since the localStorage property is - as you said - reserved by domain), but the extensions API calls this the Storage API which is divided into four buckets ("storage areas"), one of which is storage.local. So my proof-of-concept is able to use the same model across different domains (or even across extensions; see here <https://developer.chrome.com/docs/extensions/mv3/messaging/#external>). But you can use the same model to process text while browsing the web and save the results to your logged in account. In this case, you could look into the externally_connectable option ( https://developer.chrome.com/docs/extensions/mv3/messaging/#external-webpage), which allows you to receive and respond to messages from your website. For more info, see this twitter thread <https://twitter.com/clamentjohn/status/1678096766204248067>. It seems a this would require two copies of the same model being cached on the user's computer. Combining my proof-of-concept with externally_connectable should mean you only need one copy (which is stored/managed by the extension). — Reply to this email directly, view it on GitHub <#182 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFZCSEKWTUZHWPLEIJTC2QLXQQOABANCNFSM6AAAAAA2AYH5EQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>