-
Notifications
You must be signed in to change notification settings - Fork 98
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Really slow transcription on Android platform. #47
Comments
Sadly, Android was always black sheep of whisper.cpp. Check this issues: I read that some folks tried to use it on Quest 2 and also have mixed result. The only thing that I recommend is to try play with settings. The most obvious one is enable Also you can try to play around with a number of threads that you allocate for whisper. Right now it forced to use single thread, but maybe you can find some other which will work better on a quest. whisper.unity/Packages/com.whisper.unity/Runtime/WhisperParams.cs Lines 366 to 371 in c428e31
Please write back if you will have any results, I'm really interest to hear if it can run faster on quests. |
We ended up setting the threads to 3 and enabling |
Glad to hear that. Hope guys from whisper.cpp will improve Android performance in future. I'll keep this issue open for better discoverability. |
did you get an improvement on Android? |
I haven't had the time to explore potential solutions to this. However, I think the solution would have to be handled with Whisper.cpp. Looks like there have been mentions of GPU accelerated solutions. (ggerganov/whisper.cpp#1070) |
Hello,
I was recently testing this on Android and was noticing a large delay in transcription. I already tried creating a development build and adjusting the settings but got the same results. The best I could get was about 10000ms for a really short transcription. I ran a deep profile on it and noticed there was a huge delay coming from one of the 4 threads. I'm testing this on a Quest 2. It's coming from the WhisperNative.whisper_full function. Any ideas on what this could be from? There seems to be a lot of GC as well. Appreciate any pointers!
The text was updated successfully, but these errors were encountered: