encoder: default msbShift for >8-bit input#222
Open
crmurillo wants to merge 1 commit into
Open
Conversation
The compute-filter input path only shifts 10/12-bit samples into the most-significant bits (P010/X6) when input.msbShift > 0, but the default is -1. So without an explicit --msbShift, high bit-depth input reached the encoder LSB-aligned and encoded to garbage. Also add explicit --msbShift 6 to the H.265 Main10 and AV1 Main 10-bit encode samples.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The compute-filter input path only shifts 10/12-bit samples into the most-significant bits when input.msbShift > 0, but the default is -1. So without an explicit --msbShift, high bit-depth input reached the encoder LSB-aligned and encoded to garbage.
This has side-effects as well if the encoder library is used by other application like Vulkan CTS. If the application does not pass --msbShift explicitly, the result is also corrupted.
Also add explicit --msbShift 6 to the H.265 Main10 and AV1 Main 10-bit encode samples.
Type of change
bug fix
Tests
AMD Radeon RX 7600 (RADV NAVI33) / radv Mesa 26.1.3 (git-6984e91b5f) / Ubuntu 24.04.4 LTS
Total Tests: 86
Passed: 71
Crashed: 0
Failed: 0
Not Supported: 11
Skipped: 4 (in skip list)
Success Rate: 100.0%
Additional Details (optional)
The issue is reproducible running the following command:
python3 vvs_test_runner.py -t av1_main_profile_10bit --keep-filesThis will create the ivf file in:
$VVS_DIR/tests/results/test_output_av1_main_profile_10bit.ivfThe generated video
test_output_av1_main_profile_10bit.ivfcan be played back using ffplay or another application. The result is a green video.