Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request] Logging Level and Progress Bar for Model Downloads #117

Open
aress31 opened this issue May 18, 2023 · 8 comments
Open
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@aress31
Copy link

aress31 commented May 18, 2023

Right now, the output can be quite lengthy and verbose, see:

image

Could it be possible to expose logger options to granularly control the output as well as offering visual feedback for the status of the model download from Hugging Face?

@aress31 aress31 added the enhancement New feature or request label May 18, 2023
@xenova
Copy link
Collaborator

xenova commented May 19, 2023

I agree - that would be a great improvement! It was not originally considered since the library was designed for browsers, but since there has been a lot of interest for Node.js-like environments, it's definitely something to consider.

I think this would be a good first issue for someone who wants to contribute :)

@xenova xenova added help wanted Extra attention is needed good first issue Good for newcomers labels May 19, 2023
@aress31
Copy link
Author

aress31 commented May 19, 2023

That is exactly my use case, that would be for a Node application. And as such, I need a way to provide some visual feedback on the different operations, something similar in essence to:

@kungfooman
Copy link
Contributor

The screenshot log comes from WASM and you have to specify the log verbosity level in transformers.js/src/models.js, you can overwrite the constructSession function e.g. with this:

async function constructSession(pretrained_model_name_or_path, fileName, options) {
    // TODO add option for user to force specify their desired execution provider
    let modelFileName = `onnx/${fileName}${options.quantized ? '_quantized' : ''}.onnx`;
    let buffer = await getModelFile(pretrained_model_name_or_path, modelFileName, true, options);

    /** @type {InferenceSession.SessionOptions} */
    const extraSessionOptions = {
        logVerbosityLevel: 4,
        logSeverityLevel: 4,
    }
    try {
        return await InferenceSession.create(buffer, {
            executionProviders,
            ...extraSessionOptions
        });
    } catch (err) {
        // If the execution provided was only wasm, throw the error
        if (executionProviders.length === 1 && executionProviders[0] === 'wasm') {
            throw err;
        }

        console.warn(err);
        console.warn(
            'Something went wrong during model construction (most likely a missing operation). ' +
            'Using `wasm` as a fallback. '
        )
        return await InferenceSession.create(buffer, {
            executionProviders: ['wasm'],
            ...extraSessionOptions
        });
    }
}

Requires a bit discussion how the PR should then expose the session options to the transformers.js users 🤔

@xenova
Copy link
Collaborator

xenova commented May 20, 2023

A global env.logLevel should be sufficient, right?

@kungfooman
Copy link
Contributor

I kinda lend to extending the options in pipeline(), because I would rather minimize the amount of global env state/variables 🤔

https://github.com/microsoft/onnxruntime/blob/18f17c555d51caee83f15983c2620f463fbaddd1/js/common/lib/inference-session.ts#L126-L139

We could just make it a passthrough-options-ONNX-session-options-object, in the case that we need to use other session options later aswell, inside the existing pipeline options object.

Something like:

/**
 * Utility factory method to build a [`Pipeline`] object.
 *
 * @param {string} task The task of the pipeline.
 * @param {string} [model=null] The name of the pre-trained model to use. If not specified, the default model for the task will be used.
 * @param {PretrainedOptions} [options] Optional parameters for the pipeline.
 * @returns {Promise<Pipeline>} A Pipeline object for the specified task.
 * @throws {Error} If an unsupported pipeline is requested.
 */
export async function pipeline(
    task,
    model = null,
    {
        quantized = true,
        progress_callback = null,
        config = null,
        cache_dir = null,
        local_files_only = false,
        revision = 'main',
        ortExtraSessionOptions = {},
    } = {}
) {

But we could make the API easier to use aswell (e.g. using "verbose" instead of 0 because no one wants to memorize enums):

Do we even need both options, logVerbosityLevel and logSeverityLevel? It's not clear to me yet what the difference is 🙈

I have a slight feeling that other dev's are also a bit confused:

image

@xenova
Copy link
Collaborator

xenova commented May 20, 2023

Do we even need both options, logVerbosityLevel and logSeverityLevel? It's not clear to me yet what the difference is 🙈

My thought exactly. This is why a global logging level might be okay. It will also be better for when we add more backends (not just onnx). If someone REALLY wants those log levels, they can set them manually with env.backends.onnx.*

We could just make it a passthrough-options-ONNX-session-options-object, in the case that we need to use other session options later aswell, inside the existing pipeline options object.

I'm not too keen on adding that to the pipeline function, just because it's not something users will modify often. It will also then have to be used in AutoModel and similar locations.

@AmitDJagtap
Copy link

AmitDJagtap commented Jul 12, 2024

For people looking for the final answer / code. adding a nodejs code snippet here to set log level to 3 .

warning levels

VERBOSE = 0,
INFO = 1,
WARNING = 2,
ERROR = 3,
FATAL = 4

your code


import { pipeline, env } from '@xenova/transformers';
import fs from 'fs';
env.cacheDir = './.cache';
env.backends.onnx.logLevelInternal = 'error' // this line here

@fcuenya
Copy link

fcuenya commented Oct 28, 2024

Is there anything else to work on this feature? I would like to contribute but it is not clear to me if this needs further work. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

5 participants