feat(voice): implement blocking acknowledgment system#4496
Open
SheikhSarvar wants to merge 3 commits intolivekit:mainfrom
Open
feat(voice): implement blocking acknowledgment system#4496SheikhSarvar wants to merge 3 commits intolivekit:mainfrom
SheikhSarvar wants to merge 3 commits intolivekit:mainfrom
Conversation
feat(voice): implement blocking acknowledgment system - Adds wait_for_acknowledgment, set_acknowledgment, and skip_acknowledgment APIs - Implements acknowledgment logic in AgentActivity with high-priority handles - Configures automatic acknowledgment skipping on early LLM responses - Includes standard verification tests and a new example agent - Fixed potential task leaks in background speech tasks
Member
|
We already have a similar example: fast-preresponse. Not sure if this is needed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: Blocking Acknowledgment System for Voice Agents
Description
This PR introduces a "blocking acknowledgment" system to the LiveKit Agents framework. This feature allows developers to play an immediate response (e.g., "Looking that up for you...") after a user finishes speaking, ensuring the conversation feels responsive even when the LLM takes time to process.
The system intelligently manages the speech queue: if the LLM responds quickly, the acknowledgment is automatically skipped to keep the conversation fluid.
✨ Key Features
AgentSession.💡 Use Cases
RAG / Database Lookups:
When an agent needs to query a vector database or external API (which might take 1–3 seconds), playing a filler phrase like "Searching our database..." prevents dead air and confirms the agent heard the request.
Complex Reasoning:
For agents using reasoning models (like o1 or DeepSeek-R1) or multi-step tool calls that have high latency, an acknowledgment assures the user that work is in progress.
Network Latency Compensation:
In environments with variable network latency, this provides a consistent "I heard you" feedback loop regardless of backend performance.
Transaction Confirmations:
Before processing a sensitive action (like "Transferring funds"), the agent can say "One moment while I access your account..." to bridge the gap between authorization and execution.
✅ Verification
Automated Tests:
Added 4 comprehensive tests in
tests/test_acknowledgment.pycovering:All tests pass with 0 resource leaks.
Manual Verification:
Verified using the new
examples/voice_agents/acknowledgment.pyscript.