Skip to content

[stable35] fix(call): reduce virtual background performance load - #20418

Merged
Antreesy merged 4 commits into
stable35from
backport/20377/stable35
Nov 20, 2026
Merged

Antreesy merged 4 commits into
stable35from
backport/20377/stable35

Conversation

@backportbot

@backportbot backportbot Bot commented Nov 20, 2026

Copy link
Copy Markdown

Backport of PR #20377

- square model (256x256) squashed 16:9 frames by 1.78 before inference
- landscape model (256x144) matches 16:9 frames, Talk requests 1280x720 or more in calls
- model run 15.2 -> 8.9 ms with the MediaPipe CPU delegate, smaller gain on GPU
- switching models by aspect ratio is not possible
- remove unused square model

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Maksim Sukharev <antreesy.web@gmail.com>
- MediaPipe returned the confidence mask at the input frame size (1280x720)
- it uploaded the full frame, scaled the mask back up on the GPU, and the mask
  was copied between WebGL contexts at that size
- the model scales its input to 256x144 anyway, so a 256 px wide frame loses nothing
- the mask, its copies and the mask filtering now run at 256x144

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Maksim Sukharev <antreesy.web@gmail.com>
- the blur ran 3 horizontal and vertical passes at half resolution,
  each tap reading the frame and the mask
- 2 passes at quarter resolution need about 6x fewer texture reads
- the blur gets about 1.6x wider, which does not show on a blurred background
- Intel UHD 630, 720p blur: iGPU load 72 -> 64 %

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Maksim Sukharev <antreesy.web@gmail.com>
- the joint bilateral filter ran at the output size (1280x720),
  16 taps per pixel, each reading the frame and the mask
- the mask comes from a 256x144 model, half resolution keeps its detail
- blur and blend read the filtered mask with linear filtering,
  the blur pass also reads a 4x smaller texture

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Maksim Sukharev <antreesy.web@gmail.com>
@backportbot backportbot Bot added feature: frontend πŸ–ŒοΈ "Web UI" client feature: call πŸ“Ή Voice and video calls performance πŸš€ AI assisted This PR contains AI-assisted commits labels Nov 20, 2026
@backportbot backportbot Bot added this to the β›… Next Patch (35) milestone Nov 20, 2026
@Antreesy
Antreesy enabled auto-merge November 20, 2026 09:45
@Antreesy
Antreesy merged commit bf5f5f6 into stable35 Nov 20, 2026
49 checks passed
@Antreesy
Antreesy deleted the backport/20377/stable35 branch November 20, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI assisted This PR contains AI-assisted commits feature: call πŸ“Ή Voice and video calls feature: frontend πŸ–ŒοΈ "Web UI" client performance πŸš€

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant