[Feature request] Benchmark tests for CPU usage across various devices.

[MarketingPip]

Name of the feature
CPU Bench-marking

Reason for request

To investigate if a function etc is needed to be / can be improved to improve processor utilization for loading models / using models in the browser, to prevent any temporary or permanent damage to a CPU or motherboard of any device's using this library.

Additional context

Not really a "feature" but I do think this should be done as it seems when the model was fully loaded / network request completed on the device I am using caused a HUGE CPU spike.

Keep in mind - this only occurs for a brief moment when the network request / model was finished loading.

Tho when using the model - no CPU spiking.

[xenova]

Chrome dev tools will provide much better "benchmarking" functionality than anything we would be able to do. Also, hardware-level concerns that you bring up are definitely not something a JS library can control. If this was the case, you would see a ton of websites out there that could simply fry someone's computer. Rather, it is the responsibility of the web browser to handle all the low-level memory and CPU management.

Of course, we're always trying to improve the library's performance, so if you know of any other JS libraries that implement some of the functionality you are looking for, feel free to drop them in the comments :)

[MarketingPip]

@xenova fully understand it's a system / hardware level question here and not related directly to the library. Tho just figured maybe there could be potential usage / possible insight in discovering what the CPU usage & other metrics that could be measured looks like across various devices (if that makes sense).

Just a thought tho.

Didn't mean to sound like I am saying this will fry the motherboard. Just saying I would assume this thing eats CPU on some mobile devices / older devices - which I would assume is not ideal for any hardware working itself at full capacity.

That said -

Example: TensorFlow.js when loaded on a iPhone 6 (fairly old / but not Nokia stone-age old) - from my personal experience I have had the device literally crash / freeze up.

As well Transformers.js does not appear to work on the same iPhone 6 - but does not crash / freeze the device. (Tho I still need to investigate if some polyfills etc - will allow it to work on older devices)

TDLR; would be ideal if a ``system requirements``` could be determined to use Transformers.js / Transformers in the browser.

[xenova]

TDLR; would be ideal if a ``system requirements``` could be determined to use Transformers.js / Transformers in the browser.

Yeah that's definitely something we could add! Making sure these models work on phones (or notifying the user when its simply unable to) is an important goal of the library :)

[MarketingPip]

TDLR; would be ideal if a ``system requirements``` could be determined to use Transformers.js / Transformers in the browser.

Yeah that's definitely something we could add! Making sure these models work on phones (or notifying the user when its simply unable to) is an important goal of the library :)

You short summed / nailed on the head what I was trying to say hahah! 😄 I will try to see if any poly fills allow it to run on the iPhone 6 +. But I would still assume / think some great insights could come from those benchmarks (CPU / RAM usage).

[xenova]

I will try to see if any poly fills allow it to run on the iPhone.

That would be great! I've mainly been focused on getting the models on PC, but if you could help test and debug models on phones, that would seriously help! The demo site will probably be a good start, but I assume you'd need to enable developer mode (or something) to be able to view the console for any warnings or errors.

But I would still assume / think some great insights could come from those benchmarks (CPU / RAM usage).

Agreed!