Introduction

Proof-of-concept of getting OpenXR rendering support for Bevy game engine using gfx-rs abstractions.

This repository is obsolete, see https://github.com/awtterpip/bevy_openxr for current implementation

Install

Windows

Tested with Oculus Quest 2 using Oculus Link in Virtual Desktop mode (must be enabled before starting app).

Clone this repository and run install_dependencies.bat to download required dependencies (patched bevy, wgpu, openxrs crates, etc.):

git clone https://github.com/blaind/xrbevy.git
cd xrbevy
.\scripts\install_dependencies.bat

Run the example scene using Rust.

cargo run --example xr_pc_scene --release

If you don't already have an openxr loader, the process will complain about missing openxr_loader.dll file. One way to have a loader is to use a Khronos loader:

1. Navigate to https://github.com/KhronosGroup/OpenXR-SDK-Source/releases/
2. Look for the latest release and a file called openxr_loader_windows-[version].zip
3. Copy openxr_loader_windows\x64\bin\openxr_loader.dll to xrbevy folder (from zip)

The loader must also be configured to point into correct runtime. For Oculus, first install Oculus app and change the active runtime through a registry editor. Open a terminal as administrator, and run:

# Print current values
reg query HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\OpenXR\1

# Modify ActiveRuntime value to Oculus:
reg add HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\OpenXR\1 /v ActiveRuntime /d "C:\Program Files\oculus\Support\oculus-runtime\oculus_openxr_64.json"

If errors, see troubleshooting

In future, to get the latest changes:

.\scripts\update_dependencies.bat

Ubuntu

Note: tested only with Monado in virtual desktop mode, not with real hardware

Install needed dependencies:

sudo apt-get install make libudev-dev libasound2-dev

Clone this repository and run a Makefile command to download required dependencies (patched bevy, wgpu, openxrs crates, etc.):

git clone https://github.com/blaind/xrbevy.git
cd xrbevy
make download_dependencies

Run the example scene using Rust:

cargo run --example xr_pc_scene --release

If you don't already have an openxr loader, the process will complain about missing openxr_loader.dll file. Please try first One way to install a loader is to use Monado loader. Ubuntu installation:

sudo add-apt-repository ppa:monado-xr/monado
sudo apt-get update
sudo apt-get install libopenxr-loader1

If you still get the error after retrying cargo run, try:

sudo ln -s /usr/lib/x86_64-linux-gnu/libopenxr_loader.so.1 /usr/local/lib/libopenxr_loader.so && sudo ldconfig

In addition to a loader, you'll need an OpenXR runtime: either virtualized like Monado, or real like maybe (untested) SteamVR. For Monado, see https://monado.freedesktop.org/getting-started.html#monado. If you find a way to run with real hardware, please contact author or make a pull request to document the steps here.

If errors, see troubleshooting

In future, to get the latest changes:

make update_dependencies

Mac OS

Currently untested. Please make a pull request if you get it working.

Oculus Quest 2

Please note, that currently Quest 2 has a bug after exiting from app - might require restart of the device or reinstall of the app to relaunch.

See Oculus installation instructions

Background

Targets:

1. Demonstrate that OpenXR rendering is possible on Rust/GFX-RS/Bevy -ecosystem
2. Start the discussion with ecosystem participants about proper way to implement OpenXR support for Rust
3. Eventually have the basic building blocks in an easy-to-use state for building XR apps on top of bevy, or implementing XR support for other rendering engines

For technical details, see docs/architecture.md:

Bevy OpenXR plugin:

• bevy_openxr: https://github.com/blaind/bevy_openxr

Currently known pull requests for XR integration:

• bevy: bevyengine/bevy#2166 bevyengine/bevy#2319
• wgpu-rs: gfx-rs/wgpu-rs#910 (see wgpu below)
• wgpu: gfx-rs/wgpu#1387 gfx-rs/wgpu#1609
• gfx-rs: gfx-rs/gfx#3761 gfx-rs/gfx#3762

A few miscellanceous / support crates:

• bevy_rapier: dimforge/bevy_rapier#69 (only required for kinematics in the example)
• android-ndk-rs: rust-mobile/ndk#138 (needed for bundling .so for Oculus Quest 2)

Related material

Further reading - some links that have helped in getting this PoC working:

• https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html
• https://github.com/Ralith/openxrs/blob/master/openxr/examples/vulkan.rs
• https://github.com/GodotVR/godot_openxr

Community

Check out Bevy Discord channel #xr

Troubleshooting

ERROR_FORM_FACTOR_UNAVAILABLE

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ERROR_FORM_FACTOR_UNAVAILABLE', repos\gfx\src\backend\vulkan\src\xr.rs:63:14

Solution: Make sure that your device is connected

OpenXR loader not found

Could not load openxr loader. Make sure that you have openxr_loader.dll (Windows), libopenxr_loader.dylib (MacOS) or libopenxr_loader.so (Linux) in the library load path

Solution: follow the platform-specific installation instructions, and make sure that one of the above files is present in library load path (or current path).

Unknown wgpu/OpenXR extension

ADD INSTANCE EXTENSION! "VK_KHR_external_memory_capabilities"
ADD INSTANCE EXTENSION! "VK_KHR_external_fence_capabilities"
ADD INSTANCE EXTENSION! "VK_KHR_external_semaphore_capabilities"
ADD DEVICE EXTENSION! "VK_KHR_external_memory"
ADD DEVICE EXTENSION! "VK_KHR_external_memory_win32"
thread 'main' panicked at 'Unknown wgpu/OpenXR extension, add to openxr_extensions! macro: Unknown', repos\gfx\src\backend\vulkan\src\xr.rs:233:26

Solution: some of the extensions is missing from xr code. Please open a new issue to this repository, or make a pull request directly into: https://github.com/blaind/gfx/blob/bevy_openxr/src/backend/vulkan/src/xr.rs#L326 (last part with openx_extensions! macro). The format is as:

(VK_KHR_EXTERNAL_SEMAPHORE_WIN32, "VK_KHR_external_semaphore_win32", 1048576),
 ^^^ all uppercase                 ^^ format as specific by OpenXR   ^^ pow2

For the pow2 number, take the max/latest number and multiply it by two to get a next value.

No OpenXR runtime found

Error [GENERAL |  | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - failed to find active runtime file in registry
Error [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : RuntimeInterface::LoadRuntimes - unknown error
Error [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : RuntimeInterface::LoadRuntimes - failed to load a runtime
Error [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Failed to find default runtime with RuntimeInterface::LoadRuntime()
Error [GENERAL | xrEnumerateInstanceExtensionProperties | OpenXR-Loader] : Failed querying extension properties

Solution/Linux: Modify /etc/xdg/openxr/1/ files based on your runtime (see Google)

Solution/Windows: See the ActiveRuntime part of the Windows setup before