Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recognition: implement beam search for Whisper decoder #12

Open
rotemdan opened this issue Jul 27, 2023 · 0 comments
Open

Recognition: implement beam search for Whisper decoder #12

rotemdan opened this issue Jul 27, 2023 · 0 comments
Labels
enhancement New feature or request recognition Issue related to speech recognition

Comments

@rotemdan
Copy link
Member

rotemdan commented Jul 27, 2023

Beam search would enable the decoder to consider multiple recognitions simultaneously.

Currently not a high priority, because of several reasons:

  • The goal of the Whisper implementation is a good speed / quality tradeoff. Not sure having more than one decoding path would be a good tradeoff in all cases.
  • Whisper inference is currently only supported on CPU, meaning even a beam width of 2 would significantly reduce speed.
  • It is more important, at this moment, to get real-time and streaming recognition running. Due to the extra cost of beam search, it's unlikely it would be used in real-time situations (at least over CPU).
  • There are alternative approaches to get better quality, like using a larger model, or various guided decoding strategies.
@rotemdan rotemdan added enhancement New feature or request recognition Issue related to speech recognition labels Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request recognition Issue related to speech recognition
Projects
None yet
Development

No branches or pull requests

1 participant