Skip to content

Give each viewport its own say in what a track needs - #1353

Open
renefloor wants to merge 10 commits into
v2from
feat/viewport-visibility-registry
Open

renefloor wants to merge 10 commits into
v2from
feat/viewport-visibility-registry

Conversation

@renefloor

@renefloor renefloor commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Rebased on v2 now that #1351 has landed there. All five commits are this PR's own.

A participant can be drawn by several viewports at once — a tile in the grid, a picture-in-picture overlay, a livestream's host strip — while the call holds a single viewportVisibility per participant and a single subscription per track, keyed '$trackIdPrefix:$trackType'.

Every renderer wrote both directly, so whichever spoke last decided them:

  • an overlay that is not showing a participant could record them hidden — taking them out of the running for a speaker's tile and unsubscribing a track the grid is plainly drawing;
  • a small overlay that is showing them could pull a full-screen tile's subscription down to its own resolution.

#1339 worked around the first half by having a renderer re-assert upwards only — a renderer showing the participant insists, one that is not stays quiet. That stops two renderers ping-ponging, but only because it takes the argument away from one of them; it does nothing for the second half, and it is why the guards in didUpdateWidget read as an odd asymmetry.

The change

Call.viewportVisibility is a ViewportVisibilityRegistry: one ViewportMeasurement per viewport per track, and one ViewportAggregate derived from all of them.

  • visible while any viewport has the track on screen;
  • sized for the largest viewport that does — anything smaller scales down, nothing scales up;
  • persistWhenHidden if any viewport asked for it, so a screen share still overrides.

Call._applyViewportAggregate is the single writer into updateViewportVisibility / updateSubscription / removeSubscription, keeping the existing "only a remote track is subscribed to" guard. It answers whether the write landed, and the registry forgets an answer that did not — so a write that failed or was declined is driven again rather than deduplicated against.

Measuring and reporting a viewport is its own widget, ViewportVisibilityReporter: it holds the detector, the name it reports under, the deferral that keeps a report out of a build, and the release when it goes away. StreamVideoRenderer wraps its child in one and is a StatelessWidget again — there was nothing left for it to keep. The reporter is exported, so a custom widget drawing a participant's track can take part in what the call subscribes to instead of being invisible to it.

It takes a ViewportVisibilityRegistry rather than a Call, since that is all it ever used, and attach()es for a ViewportHandle of its own to report through. A reporter handed a different registry retires its old handle and attaches to the new one, rather than leaving a track recorded as on screen in a call it has left.

Two things fall out, and they are the argument for the shape:

  • The re-assert is gone. It existed so two renderers overwriting one field would settle instead of arguing. With one writer over a registry there is nothing to argue about, so the asymmetry disappears rather than being explained in a comment.
  • A new session is told every track againreapplyAll(), called at the end of join, once the join response has rebuilt the participant list. A viewport reports what changes about itself, so a tile that sat on screen across a reconnect would never speak again on its own. That dropped report is the damage #1339 was really patching over, now repaired at the source. The placement matters: a join response builds every participant afresh from the SFU carrying no viewport visibility, so anything reapplied before it is overwritten while the registry believes it was said.

Review follow-up

The fourth commit is review follow-up, on top of the shape above:

  • Call._applyViewportAggregate logs its failures, treats a disconnected call as not applied, and tells a participant the call does not have apart from a local or unpublished track.
  • The subscription is dropped on the aggregate being hidden rather than on an empty dimension, so a visible track cannot be unsubscribed.
  • ViewportVisibilityReporter ignores the last, hidden report a VisibilityDetector delivers after it is gone — dispose() there does not forget(key) — which otherwise put the viewport back with nothing left alive to release it.
  • Each reporter keys its own detector rather than keying it by track, so two viewports drawing one track cannot drop each other's reports whatever the caller passes for scopePrefix, and a reporter pointed at a new track re-reports instead of relying on the key changing. That also covers a trackIdPrefix-only change, which an SFU migration produces and the old key would not have noticed.

The viewport handle

The fifth commit is a follow-up of its own, on the registry's public shape.

A viewport used to be keyed by a String the registry minted and never recorded, so report created an entry for any string at all and release took out whichever one it was handed — and the names were '0', '1', '2'. Since the registry is exported, the invariant the class exists to protect, one measurement per viewport with none overwriting another, was stated in a doc comment and enforced nowhere.

attach() now hands out a ViewportHandle, and the handle is the key. There is no name to forge or collide with, a handle can take out only what it put in, and it belongs to the registry that made it. nextViewportId() and the viewportId: arguments are gone; the API is unreleased, so there is nothing to deprecate.

The handle also holds the track it last reported for, so reporting a new one releases the one before it and dispose() releases whatever is left. That bookkeeping used to live in the reporter's own field and its didUpdateWidget, which is why the commit is a net deletion — every consumer now gets it from the handle rather than reproducing it.

The nine reporter widget tests pass unchanged, which is the argument that this is a refactor and not a behaviour change.

Testing

  • 19 unit tests for the registry: aggregation, release, reapply for a track only now published, the reconnect flush, a viewport letting go of the track it drew before, a disposed handle saying nothing more, and an answer the call could not act on being offered again.
  • video_renderer_visibility_test.dart rewritten around what each renderer reports rather than what it writes. The load-bearing one puts the same participant in a grid tile and a smaller scrolling strip and scrolls the strip away — mutating the registry to last-writer-wins makes it fail on the visibility, and mutating it to let the last reporter size the track makes it fail on the size.
  • a renderer moved to another call stops speaking to the old one covers the registry swap: the call it leaves is told hidden, the one it joins is told visible at a real size.
  • 710 tests in stream_video, 444 in stream_video_flutter. The 38 golden failures in call_control_bar_golden_test.dart are the same ones v2 has.

One thing worth a reviewer's eye: MockCall now default-stubs viewportVisibility with a no-op registry, like it already does for isTrackPaused. Without it every widget test that pumps a tile throws, and a throw inside a VisibilityDetector callback wedges the detector for every later test in the file.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Sep 16, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: af537001-8cf8-4247-b22e-ec25d26dfe2f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.69231% with 24 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (v2@3fcff16). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...video_flutter/lib/src/renderer/video_renderer.dart 55.55% 12 Missing ⚠️
...eam_video/lib/src/models/viewport_measurement.dart 58.82% 7 Missing ⚠️
packages/stream_video/lib/src/call/call.dart 87.80% 5 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##             v2    #1353   +/-   ##
=====================================
  Coverage      ?   32.80%           
=====================================
  Files         ?      380           
  Lines         ?    29051           
  Branches      ?        0           
=====================================
  Hits          ?     9531           
  Misses        ?    19520           
  Partials      ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Base automatically changed from fix/viewport-visibility-during-build to v2 September 16, 2026 15:17
renefloor and others added 4 commits September 18, 2026 11:16
A participant can be drawn by several viewports at once — a tile in the
grid, a picture-in-picture overlay, a livestream's host strip — while
the call holds one visibility and one subscription per track. Each
renderer wrote both directly, so whichever spoke last decided them: an
overlay that was not showing a participant could record them hidden and
unsubscribe a track the grid was drawing, and a small overlay could pull
a full-screen tile's subscription down to its own size.

`Call.viewportVisibility` now holds what each viewport measures, keyed
by viewport, and derives the one answer per track: visible while any
viewport has it on screen, sized for the largest one that does. A
renderer reports only about itself and releases its measurement when it
goes away.

That makes the upward-only re-assert from #1339 unnecessary — it existed
so two renderers overwriting one field would settle rather than
ping-pong — and it lets a new session be told every track again, which
is what the re-assert was really repairing after a reconnect.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`StreamVideoRenderer` had grown a second job: drawing a track, and
measuring and reporting the viewport it draws it in. The second one is
not about video at all — anything drawing a participant's track wants
it, and nothing about it needs to know how a track is rendered.

`ViewportVisibilityReporter` takes it: the detector, the viewport's name
in the registry, the deferral that keeps a report out of a build, and
the release when it goes away. The renderer wraps its child in one, and
is a `StatelessWidget` again — there was nothing left for it to keep.

It is exported, so a custom widget drawing a participant's track can
take part in what the call subscribes to rather than being invisible
to it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It only ever reached through the call for `viewportVisibility`, and a
widget that takes the registry is a widget a test can drive without a
call at all.

The name each viewport reports under is minted by the registry now, for
the same reason: it has to be unique among the viewports reporting to
that registry, which is the only thing in a position to know. A reporter
handed a different registry releases what it measured to the old one and
reports itself to the new one, rather than leaving a track recorded as
on screen in a call it has left.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review follow-up on the viewport visibility registry.

- `reapplyAll()` moves to the end of `join`. A join response rebuilds every
  participant from the SFU carrying no viewport visibility, so reapplying
  before it left every track recorded unknown while the registry believed it
  had said otherwise.
- `_applyViewportAggregate` answers whether it landed, and the registry
  forgets an answer that did not, so a write that failed or was declined is
  driven again rather than deduplicated against. It also logs its failures,
  and tells a participant the call does not have apart from a local or
  unpublished track.
- The subscription is dropped on the aggregate being hidden rather than on an
  empty dimension, so a visible track cannot be unsubscribed.
- `ViewportVisibilityReporter` ignores the last, hidden report a detector
  delivers after it is gone, which otherwise put the viewport back into the
  registry with nothing left alive to release it.
- Each reporter keys its own visibility detector, so two drawing one track
  cannot drop each other's reports, and it re-reports when pointed at a new
  track instead of relying on the key changing.
- `ViewportMeasurement` asserts a viewport does not report `unknown`.

Tests: `Call._applyViewportAggregate` and the join wiring had no coverage,
and the two-viewport widget test compared a value to itself instead of
pinning the sizing rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@renefloor
renefloor force-pushed the feat/viewport-visibility-registry branch from bbe37d7 to 0ef026e Compare September 18, 2026 09:23
renefloor and others added 3 commits September 18, 2026 11:41
The registry keyed each viewport by a `String` it minted and never
recorded, so `report` created an entry for any string at all and `release`
took out whichever one it was handed. The names were `'0'`, `'1'`, `'2'`,
and the registry is exported — so the invariant the class exists to protect,
one measurement per viewport with none overwriting another, was stated in a
doc comment and enforced nowhere.

`attach()` now hands out a `ViewportHandle`, and the handle is the key.
There is no name to forge or collide with, a handle can take out only what
it put in, and it belongs to the registry that made it.

The handle also holds the track it last reported for, so reporting a new one
releases the one before it, and `dispose()` releases whatever is left. That
bookkeeping was spread across the reporter's own field and its
`didUpdateWidget`; every consumer now gets it from the handle instead of
reproducing it.

`nextViewportId()` and the `viewportId:` arguments are gone. The API is
unreleased, so no deprecation: the changelog entry that introduces
`Call.viewportVisibility` describes this shape.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Same behaviour, fewer words: the dartdoc and inline comments across the
registry, the handle, the reporter and `_applyViewportAggregate` said in a
paragraph what a sentence carries.

Comments only — no code changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`updateViewportVisibility` and `removeSubscription` were public because
`StreamVideoRenderer` called them from the Flutter package. It reports to
`Call.viewportVisibility` instead now, and `_applyViewportAggregate` is their
only caller, so neither is anybody else's to call.

`updateSubscription` stays public for the moment: the iOS picture-in-picture
view still subscribes through it directly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@renefloor
renefloor added this pull request to stack #1358 September 18, 2026 10:18
renefloor and others added 3 commits September 18, 2026 12:23
A viewport draws one track at a time, and reporting a new one releases the
one before it. What that costs the old track is the registry's to decide:
unsubscribed when nobody else draws it, resized down to the largest viewport
that still does.

Neither was covered at the call level, and the picture-in-picture window
switching participants is the ordinary case for it.

Also reorders the largest-viewport tests so the largest reports first. With
the smallest last, "largest wins" and "last writer wins" gave the same
answer, and the assertion held under either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`Call.leave()` clears the registry, and nothing covered it: every path out of
a disconnected call returns early whatever the registry holds, so the clear
has no consequence anyone can see while the call stays left.

The test puts the state back where a reapply would be acted on, then reapplies
and expects to hear nothing. Without the clear, a call that was left still has
a viewport standing in it and records the participant visible again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It restated the body. What a reader cannot get from the code is the return
value, so that is what is left.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@renefloor
renefloor marked this pull request as ready for review September 18, 2026 11:22
@renefloor
renefloor requested a review from a team as a code owner September 18, 2026 11:22
// now published each need the standing measurement said again.
final trackChanged = oldWidget.track != widget.track;
final publishedNow = !oldWidget.isTrackPublished && widget.isTrackPublished;
if (!registryChanged && !trackChanged && !publishedNow) return;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This guard makes persistWhenHidden a snapshot taken at report time — nothing re-reports when only that flag changes, so a runtime change to persistTrackIfNotVisible never reaches the registry. The renderer this replaces read it live, at the moment it chose between removeSubscription and updateSubscription.

ScreenShareContent flips it at runtime, off InteractiveViewer.onInteractionUpdate (persistTrackIfNotVisible: _isZoomed). Zoom into a screen share and scroll it off screen: the registry still holds persistWhenHidden: false, the aggregate comes out hidden-and-not-persisted, and the subscription is dropped — which is what the flag exists to prevent. Un-zooming is the mirror case: a stale true keeps a track subscribed that should have been let go.

One more clause covers it, and it belongs in standing's true case since what the viewport measures hasn't moved — only what it asks for:

final persistChanged =
    oldWidget.persistWhenHidden != widget.persistWhenHidden;
if (!registryChanged && !trackChanged && !publishedNow && !persistChanged) {
  return;
}

_report reads widget.persistWhenHidden live, so the deferred report picks the new value up on its own.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants