[Web] Unable to change log level and hide warning messages when creating inference session from buffer

[xenova]

Describe the issue

It is impossible to disable warning messages generated when creating an onnx session if you use a buffer to create the model (instead of a path).

Linked to:

• CleanUnusedInitializersAndNodeArgs warnings are printed only with subgraphs #14694

cc @fs-eire @guschmue

To reproduce

1. Get model files. For this example, I'll use whisper-tiny.en's decoder, but it happens for most models.

   git clone https://huggingface.co/Xenova/whisper-tiny.en

   or just download the single onnx file needed from here.

2. Run this code:

   import fs from 'fs';
   import ort from 'onnxruntime-node'
   
   const options = {
       executionProviders: ['cpu'],
       graphOptimizationLevel: 'disabled',
   }
   
   const path = './models/whisper-tiny.en/onnx/decoder_model_merged_quantized.onnx';
   const session = await ort.InferenceSession.create(path, options);
   console.log(session);
   
   // Read as buffer
   const buffer = fs.readFileSync(path);
   const sessionBroken = await ort.InferenceSession.create(buffer, options);
   console.log(sessionBroken);

3. See output:
   WORKING:

   m {
     handler: OnnxruntimeSessionHandler {
       inputNames: [...],
       outputNames: [...]
     }
   }
   

   BROKEN:

   2023-08-31 21:25:16.663056737 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/Shape_4_output_0'. It is not used by any node and should be removed from the model.
   ...
   2023-08-31 21:25:16.718106598 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.1/final_layer_norm/Constant_1_output_0'. It is not used by any node and should be removed from the model.
   m {
     handler: OnnxruntimeSessionHandler {
       inputNames: [...],
       outputNames: [...]
     }
   }
   

Urgency

Blocks the following issues from being fixed in transformers.js:

• [Feature request] Logging Level and Progress Bar for Model Downloads huggingface/transformers.js#117
• [Question] How to stop warning log huggingface/transformers.js#270

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.15.1

Execution Provider

'wasm'/'cpu' (WebAssembly CPU)

[hariharans29]

CC @fs-eire @guschmue

[fs-eire]

I build main branch and tested locally and cannot reproduce this issue. I will do a test on 1.15.1 later

[fs-eire]

I can reproduce this issue on 1.15.1 and 1.16.0. However, my local build seems has no issue. I am taking further investigations.

[fs-eire]

I found the root cause.

So this was a bug and it's fixed in main branch. However unfortunately v1.16 release does not include this change.

I will check if we are able to do this bugfix in 1.16.1 patch

[xenova]

Great, thanks for that!

[fs-eire]

v1.16.1 is released with this fix.

[wesbos]

Getting these again. Any way to filter them out? Its overwhelming my console

2024-03-01 16:26:38.495 node[30406:15443260] 2024-03-01 16:26:38.495795 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/model/decoder/layers.2/final_layer_norm/Constant_1_output_0'. It is not used by any node and should be removed from the model.

[fs-eire]

@wesbos The issue should be fixed in v1.16.1.

If you are just asking about how to deal with the warning messages, here are the solutions -

1. optimize the model: try to use the latest exporter to re-export the model, or use a model optimizer to optimize the model to remove the unused initializer. This is the way to solve the root cause.
2. set logSeverityLevel in session options to 3 or above to disallow the message being print out
3. set graphOptimizationLevel in session option to 'disabled' so that graph optimizer will not run. not recommended as this may slow down the model inferencing.

[AmitDJagtap]

For those who are still stumped, adding a nodejs code snippet here to set log level to 3 .

warning levels

VERBOSE = 0,
INFO = 1,
WARNING = 2,
ERROR = 3,
FATAL = 4

your code

import { pipeline, env } from '@xenova/transformers';
import fs from 'fs';
env.cacheDir = './.cache';
env.backends.onnx.logSeverityLevel = 3; // this line here