Web: support gradually changing sound pitch

[zommerfelds]

Hi,

I wanted to implement a nice pitch effect on a loop with hxd.snd.effect.Pitch, where de pitch value depends on some game interaction. However, the result is unusable sound skipping. I think this is due to this code:

heaps/hxd/snd/webaudio/AudioTypes.hx

Lines 52 to 53 in 8f7f439

	// BUG: Because pitch is k-rate parameter, it applies it once per 128 sample block, which throws timings off and creates audio skips.
	// Noticeable mainly with low pitch values, so it's not particularly usable to reduce pitch gradually.

So I guess it was intended for simplicity. A question though: is this a Web API limitation or just a Heaps implementation issue?

[Yanrishatum]

A bit of both. The bug described is WebAudio limitation, and I can do nothing about that. And to workaround that, we'd need to use audio worklets if we want proper sound effects (i.e. implement pitch shift manually). However that causes another issue: worklets require separate JS file as isolated context, and Heaps build system (if we can call it that) does not support having a separate .js file dedicated to audio processing.
In short:
Current solution is best I could do. In order to fix that we'd have to add worklet support to web driver and implement effects like pitch shift manually.

To elaborate on the bug comment: I have no way to catch the moment when pitch is applied, and that value directly affects how much time said 128 samples are taking up. That throws queue timing completely off. I.e. I queue sound chunk A, then chunk B, while chunk A is being played. But then user changed pitch, causing chunk A to take longer to play. I can estimate current sample somewhat accurately when that happens, and requeue chunk B at correct timing, but because it's a K-rate parameter that timing is not accurate, because if pitch was changed at sample 1, we'll have 127 sample error between chunk A and chunk B. It sure would be NICE if WebAudio had queue system that does not depend on manual timing and would allow us to just feed sequential audio data. But what can we do. It's JS.

[zommerfelds]

Thank you @Yanrishatum for the detailed answer. That makes sense, bummer!

Not sure if it's useful to keep this open, so feel free to close!

Let's see if I come up with a different idea to get a similar effect for my game. :)

[Yanrishatum]

Simplest solution would be to find a JS library that can do sound (for example Howler.js, openfl uses it iirc) and use it instead of Heaps sound system.