Migration path for OpenGL-based OSG middleware?

[fkane]

Hi, I'm the author of the SilverLining and Triton SDK's for skies, clouds, and oceans that are often used with OpenSceneGraph.

Some customers are starting to ask us about VSG integration. Admittedly I am very new to Vulkan, so forgive me if these are stupid questions.

In OSG, our integration approach was to have customers extend the Drawable interface, and then call our libraries that drew directly to whatever framebuffer was active at the time using our own OpenGL calls - being careful to reset OSG's state afterward to avoid polluting it.

We are able to produce NV_command_lists for OpenGL, and it might not be too much of a stretch to produce command buffers for Vulkan instead. But I'm not sure how this would integrate with VSG; a cursory look at the code doesn't uncover an obvious mechanism for injecting our own command buffers into VSG's. Our libraries are written with the principles of "OpenGL Like Vulkan" with the hope of making things easier when this day came.

Another approach would be to use extensions that allow interop between OpenGL and Vulkan. This is for example how we integrated with X-Plane 11.5's Vulkan renderer; they exposed a real OpenGL context synchronized with Vulkan, and we just drew to it using our existing OpenGL calls at the appropriate point. I've also seen some articles online that show how a texture rendered from OpenGL can be synchronized into Vulkan, and then perhaps used as a full-screen-quad overlay of some sort.

Has any thought been given as to how third party add-ons that need to do complex rendering of their own might work in VSG?

[vsg-dev]

One approach might be to just implement all your need to do directly in the VSG using it's built-in support for vkInstance, vkDevice, shaders, uniforms, texturing, geometry, LOD's etc, and just create a vsgSilverLinning/Triton library a bit like what @timoore has done with vsgCs integration with cesium-ion,

The standard way of implementing implementing a means of recording Vulkan commands to a command buffer is to subclass from vsg::Command and override the compile(Context&) to compile any Vulkan objects/transfer data you need to handle, and record(CommandBuffer&) and call the vkCmd*() calls from there. Potentially you could do this without using the rest of the VSG like suggested above, but you'll need to use the same vkInstance/vkDevice and manage all the rest of the Vulkan object craetion, transferring data to the GPU etc.

The VSG itself subclasses from vsg::Command to implement all the vkCmd that it supports, the implementations can be found in the directory:

https://github.com/vsg-dev/VulkanSceneGraph/tree/master/src/vsg/commands

Another approach would be to go the OpenGL integration approach, this is something that we dabbled briefly with the osg2vsg project and one of it's examples has some of the code, but we never got it far enough along to getting working properly across Windows and Linux. More time would be needed to sort that, but it sounds like you have already got that working fro X-Plane so you might be this relatively straight-forward.

The VSG exists to make using Vulkan easier, so if you haven't already written your own Vulkan code this might be the quickest route to getting good Vulkan capabilities working. The VSG is MIT licensed so you'd be completely OK to just embed the VSG in SilverLining/Triton if you wanted a way of quickly getting Vukan working, rather going through the pain of learning it from the ground up. We may need to add extra capabilities into vsg::Instance/vsg::Device to facilitate this but I think that should be possible and I'm open to such changes.

For first class VSG integration I think one may still want to provide the vsgSilvingLinning/vsgTriton approach so folks can just add nodes to the scene graph and have high quality ocean and clouds appear. Done right you could probably have another shim around these libs, or just adapter functions that exposes them in a way that could be integrated with non VSG Vulkan applications.

[timoore]

@robertosfield just weighed in with a lot of what I was going to say :) It would be interesting to see how easy it would be to embed VSG code in another non-VSG Vulkan application.

An existing example of 3rd party integration is vsgImGui, which provides a few Vulkan objects to ImGui and then lets it do the rest of the UI rendering.

If you go the OpenGL route, remember that VSG's depth buffer uses a reverse-Z scheme and would need to be flipped for most OpenGL applications.

[fkane]

Thanks so much - lots of good ideas to noodle on here.

[fkane]

Just to close the loop - we ended up subclassing vsg::Command as you suggested (as well as vsg::Group for some housekeeping stuff) and it worked! We're about to announce the launch of Vulkan support in the SilverLining sky, cloud, & weather SDK, and it's shipping with a VSG sample app that is surprisingly simple.

Thanks for pointing us in the right direction, and I think this will help with VSG adoption from our many OSG-based customers.

[robertosfield]

Congrats Frank. Glad to hear it's been straightforward to integrate.

Does the current integration support use of whole earth databases? I'm thinking of use with vsgCs or vsgExamples/data/openstreetmap.vsgt?

Another aspect I am interested in render of atmospheric effects on terrain and scene assets, this could be done as post process pass or as part of the fragment shader or a combination. I expect most 3D models will be created with the assistance of vsg::ShaderSet so would required a custom ShaderSet and extra uniforms/textures to be passed.

[fkane]

SilverLining does support whole-earth coordinate systems (i.e., ECEF) so it should work fine as long as we know which way is "up"; I haven't tried those specific examples but those would certainly be cool integrations for us to showcase next.

We don't handle shading on terrain & objects, it just draws the sky, clouds, and precipitation - and provides the simulated ambient and direct sunlight and moonlight for the given time and place so you can feed that into your own lighting. It can also generate environment maps and shadow maps based on the sky and clouds. Generally you tell SilverLining what the visibility is so it can apply it to the sky and clouds, not the other way around. That said a post-process atmospheric effect for the scene as a whole is on my to-do list as a future feature! I think that would be the only clean way to handle it given the cross-platform nature of our API.