Add Vulkan Video support

[dwlsalmeida]

Hi, I am working on this. It is 100% work-in-progress still. I am testing my changes through a RTX2060 + the proprietary NVIDIA blob on Linux and comparing the results against GStreamer's vkh264dec.

---

1. Update documentation to reflect any user-facing changes - in this repository.

2. Make sure that the changes are covered by unit-tests.

3. Run cargo fmt on the changes.

4. Please put changelog entries in the description of this Pull Request
   if knowledge of this change could be valuable to users. No need to put the
   entries to the changelog directly, they will be transferred to the changelog
   file by maintainers right after the Pull Request merge.

   Please remove any items from the template below that are not applicable.

5. Describe in common words what is the purpose of this change, related
   Github Issues, and highlight important implementation aspects.

Changelog:

### Public dependency updates
- [some_crate](https://crates.io/crates/some_crate) 1.0
 
### Breaking changes
Changes to `Foo`:
- Renamed to `Bar`.

### Additions
- Support for the `khr_foobar` extension.

### Bugs fixed
- `bar` panics when calling `foo`.

[Rua]

The Vulkan struct is named VideoProfileInfo, this one should probably match that.

[Rua]

I notice that quite a few of the types in the video.rs module have different names from their Vulkan counterparts. These should probably be renamed to match better.

Also, am I correct that your additions here only cover video decoding, and not video encoding? Encoding is still provisional in Vulkan and subject to change, so it may be better to leave that out of this PR and add it separately, or wait until it's fully accepted into the spec.

[dwlsalmeida]

Hi @Rua,

Yeah I will change the names to match better then.

If you want to, I can leave encoding out of this completely for now.

[dwlsalmeida]

@Rua How do I get these?

#define VK_STD_VULKAN_VIDEO_CODEC_H264_DECODE_SPEC_VERSION VK_STD_VULKAN_VIDEO_CODEC_H264_DECODE_API_VERSION_1_0_0

#define VK_STD_VULKAN_VIDEO_CODEC_H264_DECODE_EXTENSION_NAME "VK_STD_vulkan_video_codec_h264_decode"

Is there anything I can use in order to index into physical_device.extension_properties() ?

[Rua]

Vulkano doesn't expose those, but Ash (used by Vulkano) does. What do you need it for, though?

[dwlsalmeida]

@Rua

It is used to create the video session:

    let video_session_create_info = VideoSessionCreateInfo {
        queue_family_index: video_queue_family_index,
        video_profile: profile_info,
        picture_format: format.format,
        max_coded_extent: video_caps.max_coded_extent,
        reference_picture_format: format.format,
        max_dpb_slots: video_caps.max_dpb_slots,
        max_active_reference_pictures: video_caps.max_active_reference_pictures,
        std_header_version: video_caps.std_header_version,   <-----------------------
        ..Default::default()
    };

So far I am getting away by reusing the one returned in VkGetPhysicalDeviceVideoCapabilitiesKHR, but I wonder what if someone wants to specify a different one.

Also it's currently not possible to use vulkan_bitflags! on e.g.:

https://github.com/ash-rs/ash/blob/3f5b96b3638a9ff341584e82188c3ba3aa5cb996/ash/src/vk/native.rs#L94

This macro apparently expects the path to be available in ash::vk, but native is not reexported: https://github.com/ash-rs/ash/blob/3f5b96b3638a9ff341584e82188c3ba3aa5cb996/ash/src/vk.rs#L31

[Rua]

According to VUID-VkVideoSessionCreateInfoKHR-pStdHeaderVersion-07190, the extension name must match the one returned in the capabilities, and according to VUID-VkVideoSessionCreateInfoKHR-pStdHeaderVersion-07191, the spec version must not be greater than the one in the capabilities.

[Rua]

What do you think of making video::h264 and video::h265 submodules, to keep the specifics for these encodings separate from the generic stuff?

Regarding those constants, I'm not really sure what they are for. They don't seem to be defined in the Vulkan spec, and they're not defined in the vk.xml file either. It looks like they are actually constants defined to the video encoding scheme itself, and not by the Vulkan standard?

[dwlsalmeida]

@Rua

If you want me to I can split the codecs outside of the video module, that's fine.

Regarding the constants in native, they match the codec specifications, yes. In fact everything from the codec specs seem to be defined in ash::native instead of ash::vk. They're used to pass metadata from the codec bitstream into the driver so that the driver can program the hardware encoders/decoders in the GPU.

[Rua]

Are those constants something that the Vulkano user would be expected to use directly? If so, then they could be re-exported in Vulkano.