[Bug] Error: Unsupported model type: whisper when running in a app bundled with vite

[jgw96]

Describe the bug
A clear and concise description of what the bug is.

I am using the https://github.com/pwa-builder/pwa-starter to build my app, which uses Vite to bundle for prod builds. During dev, transformers.js works great, however, in a bundled build, I get the following error Unsupported model type: whisper when trying to make the pipeline.

My relevant code:

export async function localTranscribe(note: Note, blob: Blob): Promise<string> {
    return new Promise(async (resolve) => {
        console.log("localTranscribe", note, blob)

        if (!transcriber) {
            const { pipeline, env } = await import('@xenova/transformers');
            // @ts-ignore
            env.allowLocalModels = false;
            transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-small');
        }

        const output = await transcriber(blob, {
            language: 'en',
            task: "transcribe",
            chunk_length_s: 30,
            stride_length_s: 5,
            callback_function: callback_function, // after each generation step
            chunk_callback: chunk_callback, // after each chunk is processed
        });
        console.log('output', output);
        console.log('note', note);

        resolve(output.text);
    })
}

With some light debugging in the browser, this is the code where the error is hit.

I am assuming that this has something to do with the fact that the app is bundled at this point, but I can't seem to pinpoint the problem.

How to reproduce
Steps or a minimal working example to reproduce the behavior

Unfortunately, I dont have a repro case that I can share here. I am more looking to see if anyone else has hit this exact problem.

Logs/screenshots
If applicable, add logs/screenshots to help explain your problem.

My relevant code:

export async function localTranscribe(note: Note, blob: Blob): Promise<string> {
    return new Promise(async (resolve) => {
        console.log("localTranscribe", note, blob)

        if (!transcriber) {
            const { pipeline, env } = await import('@xenova/transformers');
            // @ts-ignore
            env.allowLocalModels = false;
            transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-small');
        }

        const output = await transcriber(blob, {
            language: 'en',
            task: "transcribe",
            chunk_length_s: 30,
            stride_length_s: 5,
            callback_function: callback_function, // after each generation step
            chunk_callback: chunk_callback, // after each chunk is processed
        });
        console.log('output', output);
        console.log('note', note);

        resolve(output.text);
    })
}

With some light debugging in the browser, this is the code where the error is hit.

Environment

• Transformers.js version: 2.6.0
• Browser (if applicable): Latest Edge Stable
• Operating system (if applicable): Windows 11
• Other: Using vite to bundle, same as the React example. however my app does not use React.

Additional context
Add any other context about the problem here.

[jgw96]

Figured this out, it was due to me using Terser as my minifier in my vite config, which I think was mangling strings. Moving back to whatever the default minifier in vite is fixed things

[xenova]

Oh that's very interesting! 👀 Thanks for reporting back. There is a mapping between strings and classes, which looks at model_type in the config.json and instantiates the corresponding class. Perhaps there is a better way to handle this which is not affected by minification.

@josephrocca has a similar problem the other day (#283), and checking the latest version, it also appears to have this problem.

I'll reopen and close when I fix this issue.

[xenova]

This PR should fix this issue. If you are able to, can you try install that branch to see if it indeed does fix your problem? I have tested locally too, and I think it should be okay, but would be good to get your feedback too!