feat(desktop): on-device speaker diarization and a People view with remembered voices

[nathanjcx]

What

On-device live transcription now tells voices apart and knows who you are, and a new People view in Memories keeps the voices Omi has learned.

Before, the local (Parakeet) path labeled by capture lane only — every mic window was SPEAKER_00 / "You" and every system-audio window "Speaker 1" — so a second person talking near the Mac was transcribed as the user (six consecutive bubbles from two people, all "You", in the report that started this).

How

Speaker identity (on-device, both lanes)

• LocalSpeakerDiarizer (actor): FluidAudio's pyannote segmentation + WeSpeaker embeddings (CoreML/ANE; FluidAudio was already a dependency) embed the dominant voice of every transcribed window. Fails open to lane labels with recordFallback(area: "stt_selection", …).
• LocalSpeakerRegistry (pure, tested): greedy cosine clustering shared by mic and system-audio lanes, running-mean centroids, matching against remembered voices, and the "who is You" policy:
  1. a remembered enrolled voice of the user locks "You" for the session; a merely learned one seeds it but stays open to correction;
  2. otherwise the dominant mic voice (speech × loudness) is the provisional user, swapped with a 1.5× margin; system-audio voices are never the user; a remote voice and its mic echo share one id;
  3. explicit paths: "This is me" in the live speaker sheet, naming a speaker (teaches that person's voice), and any push-to-talk turn ≥ 2 s (the user beyond doubt).
• Relabels move already-emitted segments in memory, in LiveTranscriptMonitor, and in SQLite via a single-statement TranscriptionStorage.relabelSpeakers (a 0↔1 swap must not double-map), queued behind pending upserts. The live name map is cleared in a pass of its own before it is rewritten, so a swap cannot let dictionary order decide which end keeps its name.
• SpeakerSegment.lane makes echo dedup lane-aware now that a speaker id no longer implies a lane; finalization no longer merges same-speaker runs backward in time (the two lanes keep independent clocks — the backend rejected end < start with a 422).
• Speaker id 0 stays the user (VoiceBargeInPolicy, EnvironmentalSpeakerContext rely on it).

Remembered voices

• LocalVoiceprintStore: per-user voiceprints.json + voice-samples/<owner>/*.wav (≤ 5 clips, ≤ 12 s each). Backward-compatible decoding.
• VoiceEnrollmentPolicy: LFU with exponential aging (one bump per app run heard, 14-day half-life), capacity 30 people, favorites pinned, user never evicted.

People view (Memories chip row, before Brain Map)

• Rows: the user first (account name, grey avatar), then favorites, then most recently talked to. Conversation count/last-talked from the local transcript store merged with the backend's conversations. Voice line and snippet chips (one per clip, play/stop) only when clips exist.
• Actions: star (pin), click-to-rename (PATCH v1/users/people/{id}/name), forget voice, delete person (backend + local voice/audio, with confirmation).
• Rebuild: pages through every conversation, fetches each detail, and for those with audio (aggregate artifact, or the single cached part placed by its first-chunk timestamp) cuts every segment, embeds it, clusters across conversations; the cluster heard in the most conversations becomes the user's voiceprint and clips, named people are rebuilt from the segments that name them. Summary reports counts, pending/locked audio, and which voice was taken as you.

Tests

New: LocalSpeakerRegistryTests (15), LocalVoiceprintStoreTests (6), VoiceEnrollmentPolicyTests (4), PeoplePageOverviewTests (3), PeopleRebuildPlannerTests (6), LocalSpeakerDiarizerRebuildTests (3), TranscriptionStorageSpeakerRelabelTests (2). Extended: LocalTranscriptionDuplicatePolicyTests (+2 lane cases), TranscriptionFinalizationStateMachineTests (+1 backward merge), navigation suites for the new hub view.

Commands run (green on the rebased branch):

• xcrun swift build -c debug --package-path Desktop
• xcrun swift test --package-path Desktop --filter "LocalSpeakerRegistryTests|LocalVoiceprintStoreTests|VoiceEnrollmentPolicyTests|PeoplePageOverviewTests|PeopleRebuildPlannerTests|TranscriptionStorageSpeakerRelabelTests|LocalTranscriptionDuplicatePolicyTests|LocalTranscriptionEndpointTests|TranscriptSpeakerAssignmentTests|TopNavigationBarLayoutTests|QueryShellTests|ChatFirstDestinationParityTests|MemoryHubSidebarRoutingTests|ChatFirstShellTests|MemoryGraphRevisitTests|TranscriptionFinalizationStateMachineTests|CaptureArchiveTests|TranscriptionStorageRecoveryTests"

Live verification (named bundle omi-speaker-diarization)

• Two synthetic voices (say -v Samantha / -v Daniel) played through the speakers got Speaker 2 and Speaker 3, consistently on the system lane and their mic echo, neither "You"; real room speech split into "You" and "Speaker 1".
• Measured WeSpeaker cosine distances: same voice 0.06–0.26 (mixed 10 s windows ≈ 0.5), different voices 0.68–0.94 — the 0.60 match threshold holds.
• The failure mode that motivated the identity work was reproduced live: a bootstrap guess stored after 20 s locked the other person in as "You" on the next launch. Guessed prints no longer lock; "This is me" / push-to-talk do.
• The first Rebuild finished in 5 s with nothing rebuilt (one page, no detail fetch, aggregate-only); the log showed it and it now walks every conversation and accepts the single cached part.
• A full Rebuild on the real account then ran end to end: 263 conversations checked · 62 with audio · 4 voices rebuilt · 11 clips saved · you = the voice in 55 of them · 8 still preparing audio, with the user cluster at 1732 s across 55 conversations out of 68 voices total — and 0 s of it carried the backend's is_user flag, which is why the most-present rule (not the flag) decides "You".

Not exercised live: person rename/delete against the backend (the endpoints' shapes were checked against backend/routers/users.py: PATCH /v1/users/people/{id}/name?value= and DELETE /v1/users/people/{id}).

Hardening after the first live run

The live Rebuild above surfaced two things a review pass then fixed, and a pre-merge review pass found a third — all three with regression tests:

• Memory. 68 distinct voices across 263 conversations, and the first cut of the rebuilder decoded each capture whole and held every embedding and clip for every voice — hundreds of megabytes on a real account. Now the decoder reads only the cut it is asked for, cluster clips are 16-bit and only the leading few voices keep any, and a cluster's voiceprint is its duration-weighted running centroid so per-cluster embedding arrays are gone.
• A rebuild used to unpin a voice. applyRebuild removed the stored row before reading it back, so a pinned person came out unpinned with their aged use score reset; it also deleted every clip before writing the rebuilt ones, so a voice the rebuild heard no audio for ended up with none — and a voice without clips shows no voice line, so it read as forgotten. The pin and the use score carry over now, and clips are only traded for clips.
• Voices followed the app, not the account. LocalSpeakerDiarizer is a singleton that resolved the signed-in user's profile once, at first touch, and was absent from the effective-owner transition's list of owner-bound local storage. Signing in as someone else kept the previous owner's prints in memory: their voices would be listed on the next owner's People page, their clips playable there, and the next owner's learned voices written back into the previous owner's file. It now retargets with RewindDatabase and the storage caches, under the transition reservation.

Follow-ups (not in this PR)

• The mic and system-audio lanes keep independent clocks (the system lane's clock only advances while audio plays), which predates this change and still defeats echo-dedup timing.
• Uploading learned voice samples to the backend's speech profiles so mobile benefits.

Failure class

The one fix: commit repairs a defect introduced earlier in this same PR (a rebuild read back the row it had just replaced), so it is not an instance of a recurring class in the registry.

Failure-Class: none

Line-count exceptions

Line-Count-Exception: desktop/macos/Desktop/Sources/AppState/AppState+Transcription.swift | 1785 -> 1803 | one helper that builds the two lane-tagged local transcription services; the diarizer logic itself lives in the new LocalTranscription/ files
Line-Count-Exception: desktop/macos/Desktop/Sources/FloatingControlBar/PushToTalkManager.swift | 4193 -> 4202 | nine lines handing the accepted turn's PCM to the diarizer as the user's voice sample
Line-Count-Exception: desktop/macos/Desktop/Sources/Rewind/UI/RewindPage.swift | 1783 -> 1788 | wires the live speaker sheet's "This is me" and naming callbacks

Product invariants affected

• INV-AUTH-1
• INV-CHAT-1
• INV-NAV-1
• INV-VOICE-1

🤖 Generated with Claude Code

https://claude.ai/code/session_01AoZMbwdDio6Vcy3xvz2CrX

[Git-on-my-level]

Thanks @nathanjcx — this is a strong piece of work, and the write-up in the description made reviewing it much easier. I went through the new diarization stack and the People view statically against the diff; nothing blocking found, and several things stood out as well done:

• LocalSpeakerDiarizer.swift: the fail-open design (models can't load → keep lane labels, record a stt_selection fallback) means the feature degrades to today's behavior instead of blank transcripts.
• TranscriptionStorage.relabelSpeakers: doing the 0↔1 speaker swap in one CASE-statement UPDATE so it can't double-map, queued behind pending upserts, pinned by TranscriptionStorageSpeakerRelabelTests — nice catch on that ordering hazard.
• LocalTranscriptionDuplicatePolicy.swift: replacing the isUser-as-lane shortcut with an explicit SpeakerSegment.lane is the right call now that a speaker id no longer implies a lane, and the new test that two mic-lane people are never deduplicated covers exactly the regression that would have been easy to miss.
• ConversationFinalizationService.mergeConsecutiveSpeakerRuns refusing to merge same-speaker runs backward in time (the two lanes keep independent clocks) — good 422 regression test.
• Test coverage overall (~1000 new lines across LocalSpeakerRegistryTests, VoiceEnrollmentPolicyTests, PeopleRebuildPlannerTests, PeoplePageOverviewTests, LocalVoiceprintStoreTests, plus updates to the existing suites) is genuinely thorough.

I also verified the backend contract the new APIClient+People.swift calls rely on exists on main today — PATCH v1/users/people/{id}/name?value= and DELETE v1/users/people/{id} (which also removes server-side speech samples) — and that FluidAudio was already a resolved dependency, so this adds no new dependencies or workflow changes.

Two things I couldn't confirm from the diff — not blockers, but worth an answer:

1. Sign-out / local data wipe. LocalVoiceprintStore writes to Application Support/users/<uid>/voiceprints.json plus voice-samples/<owner>/*.wav. Is there an existing profile-teardown path that clears that directory, or can remembered voiceprints (including third parties' voices kept by the People rebuild) outlive a sign-out? If there's no hook today, a follow-up issue may be worth it.
2. First-refresh cost. The People refresh is sequential — list → detail → signed audio download → embed per conversation, up to 2000 conversations (PeopleRebuilder.maxConversations). On a long history the first "Refresh voices" could be slow and network-heavy. Is bounding by recency planned, or is the 2000 cap considered enough?

Since this persists voice biometrics (your own and named people's — embeddings plus short WAV clips) locally and adds a new People surface to Memories navigation, it needs a maintainer's product/privacy sign-off before it lands; flagging for that rather than anything technically wrong. Also noting it's still a draft, so I'm leaving review state alone for now.

---

by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

[Git-on-my-level]

Following up on my earlier review of this head — the overall conclusion is unchanged (nothing technically blocking; it still needs a maintainer's product/privacy sign-off before it lands), and the two questions I asked there (sign-out wipe of the voiceprint store, first-refresh cost) still stand. This pass dug deeper into the rebuild/enrollment flow and turned up a few new things worth an answer:

1. The rebuild's "you" heuristic. PeopleRebuilder + VoiceClusterer.userCluster decide the user is the voice heard in the most conversations — deliberately overriding the backend's is_user flag (testTheVoiceInTheMostConversationsIsTheUser pins that) — and LocalSpeakerDiarizer.applyRebuild then replaces the existing user print wholesale, marked isEnrolled: true. An enrolled print locks "You" for subsequent sessions (reconsiderUser returns early once userIsConfirmed). For a listen-heavy user — long meetings where they speak little, lots of system-audio playback — the voice heard in the most conversations may be a remote party, and a Refresh would then hard-lock the wrong "You". Two possible softenings: restrict the user-cluster candidate to cuts whose audio originated on the mic lane when lane provenance is available, or only let a rebuilt print replace the current one when it agrees with / beats an existing enrolled print rather than always.
2. PeoplePage.isTheUser hides any person whose name equals the signed-in display name. A real contact who happens to share the user's name would silently disappear from the People list. A backend is_self flag would be sturdier when one exists.
3. Warm-up edge: while the diarizer models load (up to the 20 s grace), lane-fallback rows go out with speaker id 0 — the same id the eventual user cluster holds — and if the bootstrap later swaps ids, those warm-up rows follow the swap even if a different voice produced them. Bounded to the first ~20 s and self-corrected by later relabels, but worth knowing it's there.
4. Minor: renamePerson sends the name as a query parameter (that is the backend's current signature, so it works) — person names will land in proxy/server access logs. If the endpoint ever moves to a JSON body, both sides should move together.

None of these are blocking; #1 is the one I'd want a maintainer to weigh in on alongside the retention question, since it decides whose voice "You" means after a refresh.

— automated maintainer review on David's behalf (AI-generated)

---

by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.

[cubic-dev-ai]

27 issues found across 40 files

Confidence score: 1/5

• PushToTalkManager.swift and AppState+DataLoading.swift lack an owner-generation fence across account transitions, so detached PTT work, forgetVoice, or auth retries can write, delete, or upload voice data under the wrong account; carry the owner fence through actor mutations and authorization.
• LocalTranscriptionService.swift can remain unready indefinitely when diarizer loading stalls, disabling local transcription and cloud fallback; race model loading against an explicit timeout and handle the timeout outcome.
• LocalTranscription/LocalVoiceprintStore.swift can silently overwrite all remembered voices after a read failure or crash on incompatible embedding dimensions, causing data loss or a diarizer failure; distinguish missing files from load errors and validate or quarantine incompatible prints.
• AppState+ListenEvents.swift, LocalTranscriptionService.swift, and ConversationFinalizationService.swift have session, speaker-relabeling, and timestamp-ordering races that can apply stale actions to a new conversation or reject an upload; fence callbacks by session generation and split compaction at timestamp reversals.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="desktop/macos/Desktop/Sources/LocalTranscription/LocalSpeakerRegistry.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/LocalTranscription/LocalSpeakerRegistry.swift:122">
P1: When a system-audio window and its microphone echo have equal durations, this marks the shared remote cluster as mic-owned and can promote it to “You”. Use a strict microphone majority so equal or speaker-dominated clusters remain ineligible.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/Chat/RuntimeOwnerIdentity.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/Chat/RuntimeOwnerIdentity.swift:431">
P2: When an owner transition occurs while local transcription remains active, this call clears the diarizer's relabel sink, so a later push-to-talk enrollment still learns the voice but cannot relabel already-emitted live segments. Rebind the active sink after retargeting or stop and recreate the local transcription services as part of the transition.</violation>
</file>

<file name="desktop/macos/e2e/flows/capture-lifecycle.yaml">

<violation number="1" location="desktop/macos/e2e/flows/capture-lifecycle.yaml:17">
P2: These six `covers:` entries claim paths this flow never exercises. capture-lifecycle injects pre-formed segments via `capture_test_transcript` (bypassing LocalSpeakerDiarizer/LocalSpeakerRegistry) and never drives voiceprint persistence (LocalVoiceprintStore), enrollment (VoiceEnrollmentPolicy), rebuild (PeopleRebuilder), or the People view (PeoplePage); it only navigates to Conversations. Per the E2E coverage rule, list a source in `covers:` only when the flow actually exercises and verifies its behavior. Remove these six entries (the injected speaker_ids assert nothing about diarization output) or add steps that drive and assert these paths.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/ConversationFinalizationService.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/ConversationFinalizationService.swift:208">
P1: When a local transcript exceeds 500 runs and lane clocks move backward, compaction can combine the newly preserved runs into an invalid interval and reject the entire upload. Make compaction split at timestamp reversals or compute a valid enclosing interval for each compacted group before sending it.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/AppState/AppState+ListenEvents.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/AppState/AppState+ListenEvents.swift:244">
P1: After a stop or conversation rollover, this callback can finish against the next session and apply the old speaker action to its transcript and SQLite rows. Capture the recording/session generation before awaiting the diarizer and discard the result unless it still matches, in both identity actions and the local relabel sink.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/FloatingControlBar/PushToTalkManager.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/FloatingControlBar/PushToTalkManager.swift:1584">
P1: During an account transition, this detached task can run after `retargetEffectiveOwner` switches the diarizer to the next user's store, persisting the previous user's PTT sample and embedding as the next user's voice. Bind enrollment to an owner/session generation and reject stale work, or drain/cancel it before retargeting.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/MainWindow/TopNavigationDestinations.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/MainWindow/TopNavigationDestinations.swift:145">
P3: Adding `.people` to the `.activityChipRow` case leaves the comment above it stale: it still says "the other four hub views" though the chip row now reaches five (Conversations, Memories, Brain Map, Rewind, People). Update the comment to "five" so it stays accurate.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/MainWindow/Pages/People/PeoplePage.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/MainWindow/Pages/People/PeoplePage.swift:45">
P2: When a persisted sample file is missing or unreadable, the row still claims that clips exist. Build the caption from the successfully loaded `VoiceSnippet` values or remove invalid sample URLs before displaying it.</violation>

<violation number="2" location="desktop/macos/Desktop/Sources/MainWindow/Pages/People/PeoplePage.swift:138">
P2: When a person shares the signed-in user's display name, this name-only filter hides that person's profile. Identify the user by an explicit identity relation, or do not remove backend people solely by name.</violation>

<violation number="3" location="desktop/macos/Desktop/Sources/MainWindow/Pages/People/PeoplePage.swift:373">
P2: When focus moves to another row before this task runs, the next row overwrites the editing state and the pending rename is dropped. Capture the row and name before yielding, or serialize the commit before switching rows.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/Rewind/UI/RewindPage.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/Rewind/UI/RewindPage.swift:1609">
P2: When local STT falls back to cloud while this sheet is open, this closure remains available because the condition is evaluated when the sheet is built. It can then relabel and persist the cloud transcript using the local diarizer; recheck `useLocalSTT` inside the closure before marking the speaker.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/AppState/AppState+DataLoading.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/AppState/AppState+DataLoading.swift:275">
P1: When an account switch overlaps this mutation, the default API policy can let the request or its auth retry target the wrong owner; the generation check only suppresses local state afterward. Capture an authorization snapshot and use owner-bound policies for both person mutations.</violation>

<violation number="2" location="desktop/macos/Desktop/Sources/AppState/AppState+DataLoading.swift:285">
P1: If an account switch retargets the diarizer after the generation check but before this await, `forgetVoice` deletes that ID from the new owner’s voice store and samples. Carry the owner fence into the actor mutation or serialize deletion with owner retargeting.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/LocalTranscriptionService.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/LocalTranscriptionService.swift:150">
P1: When the diarizer model download or load stalls, this grace period never expires, so `isReady` is never set and local transcription produces no windows or cloud fallback. Race the timeout with an unstructured task that is not awaited during task-group cleanup, or independently bound the diarizer load.</violation>

<violation number="2" location="desktop/macos/Desktop/Sources/LocalTranscriptionService.swift:313">
P1: When the mic and system lanes resolve concurrently around a bootstrap speaker swap, this callback can run after the other lane has emitted an outcome based on the new ids. The old relabel map can then rewrite that segment again and leave the live transcript with incorrect speaker attribution; serialize shared diarizer outcomes with their relabel and segment delivery.</violation>
</file>

<file name="desktop/macos/Desktop/Tests/PeoplePageOverviewTests.swift">

<violation number="1" location="desktop/macos/Desktop/Tests/PeoplePageOverviewTests.swift:25">
P3: The test name promises 'then name' ordering, but no input exercises the name tie-break: only one never-heard person exists, and zed vs bob are separated by recency with coincidentally matching name order. Add two people with no activity (and unequal names) and/or two people sharing a lastTalkedAt so the localizedCaseInsensitiveCompare fallback is actually asserted, otherwise the test passes even if that branch regresses.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/LocalTranscription/LocalVoiceprintStore.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/LocalTranscription/LocalVoiceprintStore.swift:101">
P1: When an existing `voiceprints.json` cannot be read or decoded, the next voice update silently overwrites all remembered voices. Distinguish a missing file from a load failure and preserve or quarantine the old file until recovery.</violation>

<violation number="2" location="desktop/macos/Desktop/Sources/LocalTranscription/LocalVoiceprintStore.swift:142">
P2: When two clips are saved within one millisecond, this filename collides and the later WAV overwrites the earlier clip. Add a UUID to the filename so every saved sample has a distinct path.</violation>

<violation number="3" location="desktop/macos/Desktop/Sources/LocalTranscription/LocalVoiceprintStore.swift:212">
P1: A malformed or legacy voiceprint with a different embedding length crashes the diarizer on its next update with an out-of-bounds access. Validate matching dimensions and replace or skip incompatible stored prints before blending.</violation>

<violation number="4" location="desktop/macos/Desktop/Sources/LocalTranscription/LocalVoiceprintStore.swift:218">
P2: Each registry repersist sends cumulative speech time, but this branch adds it as if it were a delta. Persist only the new delta from `voiceprintsDue()` or define and honor one update-time contract, otherwise People displays inflated speech totals.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/AppState/AppState+Transcription.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/AppState/AppState+Transcription.swift:1054">
P1: After a local session ends, this sink remains installed while later cloud sessions use the same diarizer. A PTT enrollment can then relabel current cloud segments with stale local speaker IDs; guard the callback to active local transcription or clear the sink during teardown.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/LocalTranscription/PeopleRebuilder.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/LocalTranscription/PeopleRebuilder.swift:37">
P2: When a transcript segment crosses a captured-audio gap, this maps both endpoints and embeds the concatenated audio across that gap as one speaker cut. Require both endpoints to belong to the same artifact span before creating the cut.</violation>

<violation number="2" location="desktop/macos/Desktop/Sources/LocalTranscription/PeopleRebuilder.swift:38">
P2: A segment exactly one second long is rejected because subtracting `0.001` makes its mapped duration `0.999`. Preserve the exact endpoint when it is inside a span, and only use an exclusive-boundary fallback when necessary.</violation>

<violation number="3" location="desktop/macos/Desktop/Sources/LocalTranscription/PeopleRebuilder.swift:302">
P2: A rebuild over the 2,000-conversation limit retains every transcript segment, text, and translation until completion, causing avoidable memory growth. Accumulate `PersonActivity` while scanning instead of storing the full `seen` arrays.</violation>

<violation number="4" location="desktop/macos/Desktop/Sources/LocalTranscription/PeopleRebuilder.swift:379">
P1: The rebuild's heuristic `userCluster` is persisted as an enrolled voice, so one mistaken most-conversations choice locks the wrong speaker to “You” in future sessions. Preserve explicit enrollment status separately and keep this inferred user print unenrolled until the user confirms it.</violation>

<violation number="5" location="desktop/macos/Desktop/Sources/LocalTranscription/PeopleRebuilder.swift:426">
P2: For non-16-kHz source files, the one-shot reader can return truncated or empty converted ranges because it reports temporary no-data and never calls the converter again. Return `.endOfStream` after the single buffer so each cut is fully flushed.</violation>
</file>

<file name="desktop/macos/Desktop/Sources/Rewind/Core/TranscriptionStorage.swift">

<violation number="1" location="desktop/macos/Desktop/Sources/Rewind/Core/TranscriptionStorage.swift:665">
P3: personActivity() counts named speakers from all sessions, including soft-deleted ones (`deleted = 1`), whose segments are retained. A person who only ever spoke in a deleted conversation, or a person you deleted in the People view, still appears as an active recent talker. Filter `deleted = 0` on the joined session to keep activity aligned with visible memories.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: When a system-audio window and its microphone echo have equal durations, this marks the shared remote cluster as mic-owned and can promote it to “You”. Use a strict microphone majority so equal or speaker-dominated clusters remain ineligible.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/LocalTranscription/LocalSpeakerRegistry.swift, line 122:

<comment>When a system-audio window and its microphone echo have equal durations, this marks the shared remote cluster as mic-owned and can promote it to “You”. Use a strict microphone majority so equal or speaker-dominated clusters remain ineligible.</comment>

<file context>
@@ -0,0 +1,501 @@
+
+    /// A voice that is mostly heard through the speakers is a remote party, never the user —
+    /// even when its echo also reaches the mic.
+    var isMicVoice: Bool { micSeconds > 0 && micSeconds >= systemSeconds }
+  }
+
</file context>

Suggested change

	var isMicVoice: Bool { micSeconds > 0 && micSeconds >= systemSeconds }
	var isMicVoice: Bool { micSeconds > 0 && micSeconds > systemSeconds }

[cubic-dev-ai]

P1: When a local transcript exceeds 500 runs and lane clocks move backward, compaction can combine the newly preserved runs into an invalid interval and reject the entire upload. Make compaction split at timestamp reversals or compute a valid enclosing interval for each compacted group before sending it.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/ConversationFinalizationService.swift, line 208:

<comment>When a local transcript exceeds 500 runs and lane clocks move backward, compaction can combine the newly preserved runs into an invalid interval and reject the entire upload. Make compaction split at timestamp reversals or compute a valid enclosing interval for each compacted group before sending it.</comment>

<file context>
@@ -205,36 +205,18 @@ actor ConversationFinalizationService {
-          person_id: last.person_id,
-          start: last.start,
-          end: upload.end
+    let merged = Self.mergeConsecutiveSpeakerRuns(
+      bundle.segments.map { seg in
+        APIClient.UploadSegment(
</file context>

[cubic-dev-ai]

P1: After a stop or conversation rollover, this callback can finish against the next session and apply the old speaker action to its transcript and SQLite rows. Capture the recording/session generation before awaiting the diarizer and discard the result unless it still matches, in both identity actions and the local relabel sink.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/AppState/AppState+ListenEvents.swift, line 244:

<comment>After a stop or conversation rollover, this callback can finish against the next session and apply the old speaker action to its transcript and SQLite rows. Capture the recording/session generation before awaiting the diarizer and discard the result unless it still matches, in both identity actions and the local relabel sink.</comment>

<file context>
@@ -187,15 +194,84 @@ extension AppState {
+  /// remembers the voice, and every earlier bubble moves accordingly.
+  func markLiveSpeakerAsUser(_ speakerId: Int) {
+    Task { @MainActor [weak self] in
+      let relabels = await LocalSpeakerDiarizer.shared.markSpeakerAsUser(speakerId)
+      self?.applyLocalSpeakerRelabels(relabels)
+    }
</file context>

[cubic-dev-ai]

P1: During an account transition, this detached task can run after retargetEffectiveOwner switches the diarizer to the next user's store, persisting the previous user's PTT sample and embedding as the next user's voice. Bind enrollment to an owner/session generation and reject stale work, or drain/cancel it before retargeting.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/FloatingControlBar/PushToTalkManager.swift, line 1584:

<comment>During an account transition, this detached task can run after `retargetEffectiveOwner` switches the diarizer to the next user's store, persisting the previous user's PTT sample and embedding as the next user's voice. Bind enrollment to an owner/session generation and reject stale work, or drain/cancel it before retargeting.</comment>

<file context>
@@ -1574,6 +1574,15 @@ class PushToTalkManager: ObservableObject {
+      batchAudioLock.lock()
+      let userVoiceSample = batchAudioBuffer
+      batchAudioLock.unlock()
+      Task { await LocalSpeakerDiarizer.shared.enrollUserVoice(pcm16k: userVoiceSample) }
+    }
     voiceTurnCoordinator.publish(.transcriptionStarted(turnID: turnID))
</file context>

[cubic-dev-ai]

P1: If an account switch retargets the diarizer after the generation check but before this await, forgetVoice deletes that ID from the new owner’s voice store and samples. Carry the owner fence into the actor mutation or serialize deletion with owner retargeting.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/AppState/AppState+DataLoading.swift, line 285:

<comment>If an account switch retargets the diarizer after the generation check but before this await, `forgetVoice` deletes that ID from the new owner’s voice store and samples. Carry the owner fence into the actor mutation or serialize deletion with owner retargeting.</comment>

<file context>
@@ -245,6 +245,48 @@ extension AppState {
+    for (speakerId, personId) in liveSpeakerPersonMap where personId == id {
+      liveSpeakerPersonMap[speakerId] = nil
+    }
+    await LocalSpeakerDiarizer.shared.forgetVoice(personId: id)
+    log("People: Deleted person \(id)")
+    return true
</file context>

[cubic-dev-ai]

P2: When a transcript segment crosses a captured-audio gap, this maps both endpoints and embeds the concatenated audio across that gap as one speaker cut. Require both endpoints to belong to the same artifact span before creating the cut.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/LocalTranscription/PeopleRebuilder.swift, line 37:

<comment>When a transcript segment crosses a captured-audio gap, this maps both endpoints and embeds the concatenated audio across that gap as one speaker cut. Require both endpoints to belong to the same artifact span before creating the cut.</comment>

<file context>
@@ -0,0 +1,484 @@
+    let all: [PeopleRebuildCut] = segments.compactMap { segment in
+      let wallEnd = min(segment.end, segment.start + maxCutSeconds)
+      guard wallEnd - segment.start >= minCutSeconds,
+        let start = artifact.artifactOffset(forWallOffset: segment.start),
+        let end = artifact.artifactOffset(forWallOffset: wallEnd - 0.001)
+      else { return nil }
</file context>

[cubic-dev-ai]

P2: A segment exactly one second long is rejected because subtracting 0.001 makes its mapped duration 0.999. Preserve the exact endpoint when it is inside a span, and only use an exclusive-boundary fallback when necessary.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/LocalTranscription/PeopleRebuilder.swift, line 38:

<comment>A segment exactly one second long is rejected because subtracting `0.001` makes its mapped duration `0.999`. Preserve the exact endpoint when it is inside a span, and only use an exclusive-boundary fallback when necessary.</comment>

<file context>
@@ -0,0 +1,484 @@
+      let wallEnd = min(segment.end, segment.start + maxCutSeconds)
+      guard wallEnd - segment.start >= minCutSeconds,
+        let start = artifact.artifactOffset(forWallOffset: segment.start),
+        let end = artifact.artifactOffset(forWallOffset: wallEnd - 0.001)
+      else { return nil }
+      guard end - start >= minCutSeconds else { return nil }
</file context>

[cubic-dev-ai]

P3: Adding .people to the .activityChipRow case leaves the comment above it stale: it still says "the other four hub views" though the chip row now reaches five (Conversations, Memories, Brain Map, Rewind, People). Update the comment to "five" so it stays accurate.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/MainWindow/TopNavigationDestinations.swift, line 145:

<comment>Adding `.people` to the `.activityChipRow` case leaves the comment above it stale: it still says "the other four hub views" though the chip row now reaches five (Conversations, Memories, Brain Map, Rewind, People). Update the comment to "five" so it stays accurate.</comment>

<file context>
@@ -130,15 +134,15 @@ enum ShellDestination: Int, CaseIterable, Identifiable {
     /// `Activity` is what the hub's pill opens, so its door is the bar itself — the other four
     /// hub views are reached from Activity's chip row once you are there.
-    case .conversations, .memories, .brainMap, .rewind: return .activityChipRow
+    case .conversations, .memories, .brainMap, .rewind, .people: return .activityChipRow
     case .permissions: return .settingsSidebar
     case .home, .tasks, .apps, .activity: return .topBar
</file context>

[cubic-dev-ai]

P3: The test name promises 'then name' ordering, but no input exercises the name tie-break: only one never-heard person exists, and zed vs bob are separated by recency with coincidentally matching name order. Add two people with no activity (and unequal names) and/or two people sharing a lastTalkedAt so the localizedCaseInsensitiveCompare fallback is actually asserted, otherwise the test passes even if that branch regresses.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Tests/PeoplePageOverviewTests.swift, line 25:

<comment>The test name promises 'then name' ordering, but no input exercises the name tie-break: only one never-heard person exists, and zed vs bob are separated by recency with coincidentally matching name order. Add two people with no activity (and unequal names) and/or two people sharing a lastTalkedAt so the localizedCaseInsensitiveCompare fallback is actually asserted, otherwise the test passes even if that branch regresses.</comment>

<file context>
@@ -0,0 +1,66 @@
+
+    let rows = PersonOverview.ordered(people: people, activity: activity, voices: voices)
+
+    XCTAssertEqual(rows.map(\.id), ["anna", "zed", "bob", "cara"], "favorites first, then recency, then name")
+    XCTAssertTrue(rows[0].hasVoice)
+    XCTAssertTrue(rows[0].isFavorite)
</file context>

[cubic-dev-ai]

P3: personActivity() counts named speakers from all sessions, including soft-deleted ones (deleted = 1), whose segments are retained. A person who only ever spoke in a deleted conversation, or a person you deleted in the People view, still appears as an active recent talker. Filter deleted = 0 on the joined session to keep activity aligned with visible memories.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At desktop/macos/Desktop/Sources/Rewind/Core/TranscriptionStorage.swift, line 665:

<comment>personActivity() counts named speakers from all sessions, including soft-deleted ones (`deleted = 1`), whose segments are retained. A person who only ever spoke in a deleted conversation, or a person you deleted in the People view, still appears as an active recent talker. Filter `deleted = 0` on the joined session to keep activity aligned with visible memories.</comment>

<file context>
@@ -617,6 +617,68 @@ actor TranscriptionStorage {
+      let rows = try Row.fetchAll(
+        database,
+        sql: """
+          SELECT s.personId AS personId, COUNT(DISTINCT s.sessionId) AS sessions, MAX(t.startedAt) AS lastStartedAt
+          FROM transcription_segments s
+          JOIN transcription_sessions t ON t.id = s.sessionId
</file context>

[Git-on-my-level]

Reviewed the rewritten branch (head 3061d15) against my earlier reviews. The rewrite landed four things I verified in the diff, three of them answering real findings:

• Remembered voices now follow the account: RuntimeOwnerIdentity.swift retargets LocalSpeakerDiarizer under the owner-transition reservation, and LocalVoiceprintStore.forCurrentUser resolves an empty owner id to the anonymous profile the same way RewindDatabase does — the cross-account voiceprint leak described in the PR notes is closed for the in-memory store.
• A rebuild no longer unpins or mutes a voice: applyRebuild carries the pin and the decayed use score over, and a voice the rebuild heard no audio for keeps its previous clips (pinned in LocalSpeakerDiarizerRebuildTests).
• Rebuild memory is now bounded: AudioClipDecoder.Reader decodes only the cut it is asked for, and VoiceClusterer keeps 16-bit clips with only the leading few clusters holding audio at all.
• applyLocalSpeakerRelabels clears the live name map in a pass of its own before writing, so a 0↔1 swap can no longer lose a name to dictionary iteration order.

Still standing from my earlier reviews, none blocking but each worth a maintainer's call:

1. Retention on exit. The owner transition swaps the in-memory store, but I still can't find anything that deletes users//voiceprints.json or the voice-samples/ clips on sign-out or app-account deletion. Is there a profile-teardown hook these should join, or is a follow-up issue warranted? These files are voice biometrics of the user and of named third parties.
2. The rebuild's "You" heuristic. VoiceClusterer.userCluster picks the voice heard in the most conversations and applyRebuild replaces the user's print wholesale, marked enrolled — which locks "You" for subsequent sessions. For a listen-heavy user (long meetings where they speak little, lots of system-audio playback) the most-present voice may be a remote party. Two possible softenings stand: restrict the user cluster to mic-lane-provenance cuts when lane provenance is available, or let a rebuilt print replace an existing enrolled one only when it agrees rather than always.
3. PeoplePage.isTheUser hides any person whose name equals the signed-in display name, so a real contact who shares the user's name silently disappears from the People list. A backend is_self flag would be sturdier when one exists.
4. Minor, unchanged: renamePerson sends the name as a query parameter (that is the backend's current PATCH signature); person names will land in proxy/server access logs. If the endpoint ever moves to a body, both sides should move together.

Nothing technically blocking on this head, and the engineering quality is genuinely strong — the pure LocalSpeakerRegistry with synthetic-embedding tests, the single-statement CASE relabel that cannot double-map a 0↔1 swap, lane-based echo dedup with the isUser fallback for pre-migration rows, and the backward-merge guard that fixes the end<start 422 are all pinned by tests. Desktop build/tests and static contracts are green.

Because this persists voice biometrics locally (yours and named people's — embeddings plus short WAV clips) and adds a new People surface to Memories, it needs a maintainer's privacy/product sign-off before merge; item 1 above is the concrete decision I'd start with.

— automated maintainer review on David's behalf (AI-generated)

---

by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.