feat(linux): Add Vulkan video encoder for Linux #4603
Conversation
- Add FFmpeg Vulkan encoder with zero-copy DMA-BUF/EGL interop - Add kms_vblank option for vblank-synchronized capture - Fix KMS framebuffer scaling for HiDPI displays - Add Vulkan encoder settings to web UI - Add documentation for vulkan encoder and kms_vblank options
neatnoise
changed the title
neatnoise
changed the title
There was a problem hiding this comment.
This is just a preliminary review. On top of the comments below, I think these files will probably need updates.
update dependencies:
- https://github.com/LizardByte/Sunshine/blob/master/scripts/linux_build.sh
- https://github.com/LizardByte/Sunshine/blob/master/packaging/sunshine.rb
- https://github.com/LizardByte/Sunshine/blob/master/packaging/linux/Arch/PKGBUILD
- https://github.com/LizardByte/Sunshine/blob/master/packaging/linux/copr/Sunshine.spec
- https://github.com/LizardByte/Sunshine/blob/master/.github/workflows/ci-freebsd.yml
- and maybe the flatpak
There was a problem hiding this comment.
When adding/updating localization, only this file should have updates. The other files should be left untouched. It will be handled automatically by CrowdIn.
| # vulkan video encoding (via FFmpeg) | ||
| find_package(Vulkan QUIET) | ||
| if(Vulkan_FOUND) | ||
| add_compile_definitions(SUNSHINE_BUILD_VULKAN) | ||
| include_directories(SYSTEM ${Vulkan_INCLUDE_DIRS}) | ||
| list(APPEND PLATFORM_LIBRARIES ${Vulkan_LIBRARIES}) | ||
| list(APPEND PLATFORM_TARGET_FILES | ||
| "${CMAKE_SOURCE_DIR}/src/platform/linux/vulkan_encode.h" | ||
| "${CMAKE_SOURCE_DIR}/src/platform/linux/vulkan_encode.cpp") | ||
| endif() |
There was a problem hiding this comment.
This pattern is wrong. Check the wayland section just below.
We use a cmake option to indicate if the feature should be on or off. It should be on by default. If the required packages are not found it should fail the build.
| <tr> | ||
| <td>vulkan</td> | ||
| <td>Use FFmpeg Vulkan encoder with zero-copy DMA-BUF (AMD, Intel, NVIDIA on Linux). | ||
| Requires FFmpeg built with Vulkan support. | ||
| @note{Applies to Linux only.}</td> | ||
| </tr> |
There was a problem hiding this comment.
| <tr> | |
| <td>vulkan</td> | |
| <td>Use FFmpeg Vulkan encoder with zero-copy DMA-BUF (AMD, Intel, NVIDIA on Linux). | |
| Requires FFmpeg built with Vulkan support. | |
| @note{Applies to Linux only.}</td> | |
| </tr> | |
| <tr> | |
| <td>vulkan</td> | |
| <td>Use FFmpeg Vulkan encoder with zero-copy DMA-BUF (AMD, Intel, NVIDIA on Linux). | |
| @note{Applies to Linux only.}</td> | |
| </tr> |
This is too much info for users, and will probably confuse them.
Edit: to add clarification, users do not need to install FFmpeg, as this is a build only dependency.
| # vulkan video encoding (via FFmpeg) | ||
| find_package(Vulkan QUIET) | ||
| if(Vulkan_FOUND) | ||
| add_compile_definitions(SUNSHINE_BUILD_VULKAN) |
There was a problem hiding this comment.
We use a SUNSHINE_DEFINITIONS list for this. This way it carries over to the test binary build.
| #include "src/utility.h" | ||
| #include "src/video.h" | ||
| #include "vaapi.h" | ||
| #ifdef SUNSHINE_BUILD_VULKAN |
There was a problem hiding this comment.
can we avoid the macro for the include?
|
|
I will convert this to draft for now, please change it back when you're ready for a full review. Thanks! |
|
Its ready |
|
I know this PR is about encoding, but I saw changes to capture as well. Do you know if this could support variable framerate capture, similar to how capture works on Windows? In my limited experience Linux always captures at the stream framerate. The ideal capture process would be capturing only when the image has changed (with duplicated frames subject to minimum_fps_target), as well as capturing using a frame interval timer requested by the client or based on the framerate. For example, an Xbox client works best if frames are captured on a timer with an interval of 59.94fps (60000/1001), a desktop client could just use a 60fps interval. One thing you do not want to do is base anything on the host's vsync interval. |
I left a bunch of review comments. |
3 participants
Description
This PR adds Vulkan video encoding support for Linux KMS capture, providing an alternative to VAAPI encoding. AI agent was used heavily for code generation, following the existing VAAPI encoder implementation as the reference pattern.
Changes:
h264_vulkan,hevc_vulkan,av1_vulkan) with zero-copy DMA-BUF import via EGL interop - no GPU-to-CPU copies in the capture-encode pipelinefb->width/fb->height) instead of logical viewport dimensions, fixing capture on HiDPI displaysvulkanencoder andkms_vblankoptions indocs/configuration.mdDependencies:
Requires corresponding PR in LizardByte/build-deps (
ffmpeg-vulkanbranch) to enableBUILD_FFMPEG_VULKANoption, which adds--enable-vulkanand Vulkan encoder support to FFmpeg builds.Testing:
vulkan-headersandvulkan-icd-loaderpackages to be installed (or similar on another distros).Screenshot
Issues Fixed or Closed
Roadmap Issues
Type of Change
Checklist
AI Usage