On Oculus Quest: 6 stale frames per second

[kennardconsulting]

Hardware

Oculus Quest

Steps to Reproduce

First:

• Visit http://moonrider.xyz using the Oculus Browser on Quest
• Click to enter VR (glasses icon in bottom right corner)
• Bend your knees. Move your whole torso from left to right (i.e. like a 'strafing' action, not turning your head)
• Notice the edges of things like the 'Start' button and the grey star you are standing on (look down). They move in a choppy way.

Then Compare:

• Open Beat Saber demo (native app) and start tutorial
• Move to the corner of the black square you are standing on (with the white edge) and look down
• Repeat same left/right strafing movement
• Notice the white edges of the square. Notice movement is not choppy.

Notes

Is not Moon Rider specific. Is easily seen on most AFrame demos, such as Anime-UI (https://aframe.io/aframe/examples/showcase/anime-UI) and Towers

I believe this is a bug in the positional tracking. UPDATE: a bug related to stale frames (see below)

Current Behavior

Positional tracking is choppy

Expected Behavior

Positional tracking is as good as native apps (such as Beat Saber, and the Firefox Reality UI)

Possible Solution

This issue also affects the Oculus Browser. But it does not affect Exokit Native running a WebVR app.

Context

​It has a significant impact on comfort for anything less than 2 metres away.

Error Logs and Stack Traces

Other reports:

https://www.reddit.com/r/OculusQuest/comments/c6p3d5/jitter_judder_stutter_in_the_oculus_browser/?utm_source=share&utm_medium=ios_app

[philip-lamb]

@fernandojsg This sounds like a content bug and it mentions A-Frame. Do you have any comment?

[philip-lamb]

/remind me in 2 days close if no response.

[reminders]

@philip-lamb set a reminder for Jul 19th 2019

[kennardconsulting]

Thanks for your response!

I do not believe this is a content bug, since the same content is choppy on Oculus Browser and Firefox Reality, but runs smooth on Exokit native. See:

"I tried https://webvr.info/samples/03-vr-presentation.html on Quest Oculus Browser and I am seeing visual frame misses... I tried the same code in Exokit native and it's buttery smooth"
aframevr/aframe#4222 (comment)

[reminders]

👋 @philip-lamb, close if no response.

[kennardconsulting]

I don't think it's content related, because it works in Exokit Native. Please don't close :)

[fernandojsg]

maybe the issues on the aframe demos like moon-rider is because they are not hitting the correct framerate and it's dropping frames there instead of a bad positional tracking? I'll try the webvr.info demo to see if I can reproduce it, while checking the stats from oculus, if it's dropping frames it's definitely a different problem than pos tracking.
Thanks for reporting

[kennardconsulting]

100% agree if it's dropping frames (either Moon Rider or one of the other demos) then it's a content issue. But when I check webvr.info/samples/03-vr-presentation.html it stays at a solid 72fps and the issue is clearly visible when looking at the cubes and strafing your torso left-to-right.

Huge thanks for looking into it further. Much appreciated for our project.

[kennardconsulting]

Hi @fernandojsg. Did you have any success reproducing this issue?

I've noticed that the same choppy-ness appears even if you just sit still and watch the moving cube that is flying around in the webvr.info demo. So perhaps this is not a positional tracking issue after all (sorry) but more of a frame timing issue?

If anything, this would make the issue more serious as it would affect both 6DoF and 3DoF applications.

[MortimerGoro]

I can reproduce the issue in the Quest. I found that there are about 6 stale frames per second according to Oculus stats. Probably related to that? Oculus Browser seems to have a similar problem, the moving cube is choppy too.

I'm going to test how #1429 affects the stale frames. It should help as we'll have less buffer copies.

[fernandojsg]

@MortimerGoro can you give it a try on the Focus too?

[kennardconsulting]

@MortimerGoro so happy to have independent confirmation! This is a really big deal to us. Thanks so much to you and @fernandojsg for looking into it.

[kennardconsulting]

I can confirm this works fine in Exokit Native. You can try this for yourself by:

1. Downloading https://transfer.sh/ErXjp/app-debug.apk
2. adb install -r app-debug.apk
3. adb shell am start -n com.webmr.exokit/android.app.NativeActivity -e ARGS "'node --experimental-vm-modules /package -x webvr https://webvr.info/samples/03-vr-presentation.html'"

On Oculus Quest, the moving cube and the positional tracking are smooth, not choppy

[kennardconsulting]

@MortimerGoro can you confirm if this means Exokit Native shows no stale frames? Is that the cause of the problem?

[MortimerGoro]

@kennardconsulting Thanks for sharing the APK. I tried it and it's pretty stable. It has 0 stale frames in the Oculus Go most of the time.

Yep, I think the problem is related to stale frames. I'm doing some research to see how we can reduce them. On full-featured browsers we have a lot more layers and security restrictions than Exokit, so it makes it harder to achieve the perfect latency

[kennardconsulting]

I understand. I appreciate your efforts!

[MortimerGoro]

Some findings from the research:

• Stale frames can be caused by GC and I found some Float32Arrays are recreated in Gecko. Posted a patch https://bugzilla.mozilla.org/show_bug.cgi?id=1569605
• elgSwapBuffer call to copy WebGL canvas frame to the WebVR Surface sometimes takes more time causing stale frame. This PR from @daoshengmu should help fix the issue: Render WebVR frames to OVR VR surfaces #1429
• I've found some delays at the end of https://searchfox.org/mozilla-central/source/gfx/vr/ipc/VRManagerChild.cpp#368 C++ to JS call. I've been chatting with the JS team and we'll try to capture a profile of the specific frame with the delay to better understand the problem

[kennardconsulting]

@MortimerGoro can you please comment on this issue when there is some version of an APK that contains 1 or more of these fixes? Happy to test it for you!

[daoshengmu]

@kennardconsulting I don't see noticeable chopping after applying the patch of #1429. But, I found the framebuffer will get stuck when hitting objects in the gameplay at the first time. Is it a known issue?

[kennardconsulting]

I doubt it. MoonRider has had a lot of testing.

[kennardconsulting]

Can you try https://webvr.info/samples/03-vr-presentation.html instead?

[daoshengmu]

Can you try https://webvr.info/samples/03-vr-presentation.html instead?

We will keep staying 72 FPS in the immersive mode on Quest.

[kennardconsulting]

Sorry I don't understand your response. I was asking if there was "noticeable chopping" in 03-vr-presentation.html

Are any of the changes above in the 1.4 FxR release?

• Float32Arrays fix
• elgSwapBuffer fix
• delays at the end VRManagerChild.cpp fix

With the 1.4 FxR release I still see chopping (although it does, subjectively, seem reduced)

[philip-lamb]

Just the first one so far. Reduction in buffer copies is coming soon with Daosheng's work.

[kennardconsulting]

@philip-lamb okay thanks. Definitely moving in the right direction though!

[kennardconsulting]

Please see this wonderful bug report from @Galadirith which demonstrates this issue: aframevr/aframe#4222 (comment)

[kennardconsulting]

Also this preliminary investigation by the Oculus Team: aframevr/aframe#4222 (comment)

[kennardconsulting]

Also this finding from Oculus team: "After more investigation, found out that VSyncs are delivered quite unstably. Currently, WebVR/XR in Oculus Browser is using Android Choreographer to generate VSyncs and seems like this is not the best way. Probably, will use 'end-of-frame' signal from our VrDriver instead."

[kennardconsulting]

Does Firefox Reality also use Android Choreographer to generate VSyncs?

[kennardconsulting]

Update: Oculus confirms that switch to VrDriver for V-Sync helps a lot.

Also "While working on this issue, I found another one: a separate issue of controllers-related jitter. The issue is that controller's poses are predicted for the certain frame, while gamepad polling logic has no idea which pose it is picking up at the moment. So, sometimes it picked up wrong poses for the controllers which caused jitter for the controllers' rendering (while the rest of animation / head movement is still smooth). I am working on fixing that."

Does any of this apply to Firefox Reality?

[IvoJager]

Are there any updates regarding this? I can confirm the same problem on Vive Focus (non plus) and Oculus Go. The Oculus Browser (tested on Go) is silky smooth, while the same scenes on Firefox Reality are, quite frankly, universally unsable and nausea inducing - definitely not where I'd like Firefox Reality to be!

All AFRAME scenes display the same issue for me on Go and Focus. The AFRAME Snowglobe example is a particularly good example that shows dropped frames and copious amounts of stutter on both devices.

EDIT: Also, given the severity of this (VR browser unable to render VR experiences), I would have thought a higher priority tag for this bug would have been in order...

[kennardconsulting]

Oculus Quest Browser 7 (currently in beta branch) is very smooth now. Thanks to the V-Sync fix, some fix to controller prediction, and other improvements. Hoping Firefox Reality will follow suit.

[MortimerGoro]

Have you tried https://webvr.info/samples/03-vr-presentation.html in the latest builds? I see pretty stable 0 stale frames now in that sample. Some of the fixes mentioned have been already landed (still waiting for #1429) in addition to using the latest Oculus SDKs.

[IvoJager]

@MortimerGoro thank you. The vr presentation sample indeed works smoothly on the Vive Focus with the 2019-12-03 build with 0 stale frames. I am still running into severe stutter and performance issue in some AFRAME scenes compared to Oculus browser however. I will do some more testing/sleuthing.