Does this model support streaming?

[AnkushMalaker]

If so, how is the performance compared to recently released https://github.com/revdotcom/reverb or https://github.com/ufal/whisper_streaming?
And can you provide an example snippet or script?

If not, I'm a little confused, acc to the blog post, a huge focus was to run this on edge devices like raspberry pis and microcontrollers where streaming becomes important.
Thanks for your awesome work!

[guynich]

@AnkushMalaker
Moonshine is a batch model. We provide the open-source demo live_captions.py as an example of how to implement repeated model inferences with live speech. We have not quantified performance of our demo against other methods such as your links. I believe those may also use batch models at the core too, such as Whisper, with possibly similar wrapper approaches to enable streaming-like operation. Thanks for your interest.

[keveman]

Yes, we are running Moonshine on Torre. Currently we have tiny and base versions of Moonshine that have equal or better accuracies compared to Whisper's tiny.en and base.en models respectively, but run multiple times faster for short audio sequences. We do have onnx versions available : https://github.com/usefulsensors/moonshine/blob/main/moonshine/demo/onnx_standalone.py

[nirbhaysinghnarang]

I understand that. However, what I was getting at is more along the lines of asking if you have any plans of supporting the small.en equivalent?

[nirbhaysinghnarang]

Interesting that you are running Moonshine on Torre. The demo indicates real-time transcription - are you following the algorithm outlined in demos/live_captions.py i.e. using VAD to detect utterances? If so, I would suggest doing something a little different. The fine folks at sherpa-onnx have a really solid 2-pass implementation where they use a pretty inaccurate online, streaming model with endpoint detection to not only detect utterances (which are then passed into an offline model like moonshine or whisper) but also give interim updates.

[keveman]

The 2 pass method used in sherpa-onnx is indeed interesting. However, in our trials, using a 20M zipformer as the first pass caused pretty bad first pass output, which is not surprising. It eventually got all fixed up correctly by the second pass (whisper base in our trials). In our implementation, we use moonshine itself for both passes and silero-vad to detect pauses. Empirically, the quality looked better because we are using the same high accuracy model for both passes. However, I confess the quality of output from our streaming implementation is not as good as it can be and we are constantly improving it.

[AnkushMalaker]

Its hard to use if streaming performance is not good. For any application where you'd want STT on edge (home assistant/voice assistant), it depends on good streaming performance.
I suppose this is still good for replacing whisper for large scale ASR. Helps the pockets. Thanks for your work.

[csukuangfj]

For those of you who want to try the two-pass method with moonshine models in sherpa-onnx, please download pre-built Android APKs from
https://k2-fsa.github.io/sherpa/onnx/android/apk-2pass.html

---

APKs for silero_vad + moonshine models are at
https://k2-fsa.github.io/sherpa/onnx/vad/apk-asr.html

[nirbhaysinghnarang]

Is the code for these demos open-sourced?

[csukuangfj]

Yes, everything from Next-gen Kaldi is open-sourced, including sherpa-onnx.

please see
https://github.com/k2-fsa/sherpa-onnx/tree/master/android

[csukuangfj]

Hint: Doc is at https://k2-fsa.github.io/sherpa/onnx