Twilio Agent Connect

A powerful SDK for building intelligent, context-aware AI agents with Twilio's communication technologies.

Documentation ◆ API Reference ◆ Python SDK ◆ TypeScript SDK ◆ Examples

Seamlessly integrate with Twilio Conversation Memory and Conversation Orchestrator to build LLM-powered agents with persistent memory and conversation context.

Tip

Building AI agents on AWS, Microsoft, or Google? Connect them to Twilio's voice, messaging, and conversation context with these dedicated packages:

• TAC for AWS — Strands, Bedrock Agents, Bedrock AgentCore
• TAC for Microsoft — Microsoft Agent Framework, Azure AI Foundry (incl. Voice Live), Azure OpenAI
• TAC for Google — ADK and Agent Studio on Agent Engine, CX Agent Studio, Conversational Agents (Dialogflow CX)

---

Key Features

• Multi-Channel Support: Built-in handling for Voice (ConversationRelay), SMS, RCS, WhatsApp, and Chat
• Multi-Sender Support: Configure multiple phone numbers, RCS Sender IDs, or WhatsApp numbers per channel by setting the comma-separated plural env var (TWILIO_PHONE_NUMBERS, TWILIO_RCS_SENDER_IDS, TWILIO_WHATSAPP_NUMBERS) instead of the singular one; the first entry is the default sender for outbound
• Outbound Conversations: Agent-initiated conversations across all supported channels
• ConversationRelay-Only Mode: Get started quickly with TAC's voice plumbing (TwiML, WebSocket, callbacks) before adding Conversation Orchestrator or Conversation Memory
• Memory Management: Automatic integration with Twilio Conversation Memory for persistent user context
• Conversation Lifecycle: Automatic tracking of conversation sessions and state
• Human Handoff: Built-in tool to route conversations to human agents via Twilio Studio Flows (including Flex)
• Horizontal Scaling: Run N replicas behind a load balancer with no shared datastore; for voice, set TWILIO_INSTANCE_PUBLIC_DOMAIN to each replica's own public hostname

Get Started

To get started, set up your Node.js environment (Node.js 22.13.0 or newer required), and then install the TAC SDK package:

npm install twilio-agent-connect

Quick Examples

Option 1: Use the Setup Wizard

Use the Twilio Setup Wizard to automatically create a Memory Store and Conversation Configuration and generate your .env file:

git clone https://github.com/twilio/twilio-agent-connect-typescript.git
cd twilio-agent-connect-typescript
npm install
npm run setup  # Open http://localhost:8080

Option 2: Manual Setup

You can also create a Memory Store and Conversation Configuration manually through the Twilio Console. For a full walkthrough — credentials, Console navigation, and webhook configuration — see the TAC Quickstart.

---

After completing setup, here's a minimal example to get started:

Multi-Channel with OpenAI SDK

Use the OpenAI SDK to build an AI agent that works across Voice and SMS channels with conversation memory and user context.

First, install the required dependencies:

npm install twilio-agent-connect openai dotenv

Note: dotenv is optional — TAC works with environment variables from any source (.env files, Docker, Kubernetes, CI/CD, shell exports, etc.).

Then create your application:

import { config } from 'dotenv';
import OpenAI from 'openai';
import {
  TAC,
  TACConfig,
  VoiceChannel,
  SMSChannel,
  TACServer,
  MemoryPromptBuilder,
} from 'twilio-agent-connect';

config();

const openai = new OpenAI();

// Initialize TAC and channels
const tac = await TAC.create({ config: TACConfig.fromEnv() });
const voiceChannel = new VoiceChannel(tac);
const smsChannel = new SMSChannel(tac);

// Store conversation history
const conversationHistory: Record<string, OpenAI.Chat.ChatCompletionMessageParam[]> = {};

// System instructions for the AI agent
const SYSTEM_INSTRUCTIONS =
  'You are a customer service agent speaking with a user over voice or SMS. ' +
  'Keep responses short and conversational — a sentence or two. ' +
  'Do not use markdown, asterisks, bullets, or emojis; your words will be ' +
  'spoken aloud or sent as plain text.';

// Handle incoming messages
tac.onMessageReady(async ({ conversationId, message, memory, session }) => {
  const convId = conversationId as string;

  if (!conversationHistory[convId]) {
    conversationHistory[convId] = [];
  }

  // Build system prompt with memory context using compose()
  const systemPrompt = MemoryPromptBuilder.compose(SYSTEM_INSTRUCTIONS, memory, session);

  conversationHistory[convId].push({ role: 'user', content: message });

  const response = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      { role: 'system', content: systemPrompt },
      ...conversationHistory[convId],
    ],
  });

  const llmResponse = response.choices[0]?.message?.content ?? '';
  conversationHistory[convId].push({ role: 'assistant', content: llmResponse });

  return llmResponse;
});

const server = new TACServer(tac, { voiceChannel, messagingChannels: [smsChannel] });
await server.start();

Note: See the getting started guide for complete setup instructions and .env configuration details.

That's it! The server automatically:

• Creates Fastify app with /twiml, /ws, and /webhook endpoints
• Handles Voice and SMS conversations
• Routes responses to the appropriate channel
• Provides conversation memory and user profile in the callback

For configuration details and environment variables, see the getting started guide.

How It Works

TAC simplifies building AI agents by handling the integration between Twilio's communication channels and your LLM:

Message Flow

1. Webhook/Connection Received: Twilio sends webhook (SMS) or WebSocket connection (Voice) to your server
2. Channel Processing: Channel validates and processes the incoming event
3. Memory Retrieval: TAC optionally retrieves user memories and profile from Memory
4. Callback Invoked: Your onMessageReady callback receives user message, context, and optional memory response
5. Response Handling: Your callback returns a response string that TAC routes to the appropriate channel

For detailed architecture and advanced usage, see CLAUDE.md.

Telemetry

The SDK reports usage events to help improve the product — conversation started and ended, messages received, responses sent, and voice connection and interrupt events. Each event includes your Twilio account SID, the channel (when TAC already knows it — conversation ended events from Conversation Orchestrator don't carry one), a conversation ID, the SDK name and version, and timing such as conversation duration.

Message content, transcripts, phone numbers, and other end-user identifiers are never collected.

Telemetry is on by default. To disable it, set the following environment variable:

TAC_ANALYTICS_DISABLED=true

Learn More

Examples & Guides:

• API Reference - Full API documentation generated from the source
• Getting Started Guide - Examples and comprehensive documentation
• Twilio Setup Wizard - Create your Memory Store and Conversation Configuration from a local web UI
• OpenAI SDK Example - Complete multi-channel example with Voice, SMS, and Chat
• WhatsApp Example - WhatsApp channel with memory integration
• Chat Example - Web chat integration example
• ConversationRelay-Only Mode - Get started with voice using just ConversationRelay
• OpenAI Realtime over Media Streams - Speech-to-speech voice calls, bridging Twilio Media Streams to the OpenAI Realtime API
• GPT-Live over Media Streams - Speech-to-speech voice calls, bridging Twilio Media Streams to the OpenAI GPT-Live API
• Outbound Conversations - Agent-initiated conversations example
• Voice Call Events - Answering machine detection, recording, and call disposition on outbound calls
• Voice DTMF - Keypad input over ConversationRelay — collect an account number digit by digit and hand it to the agent as context
• More examples coming soon

AWS, Microsoft, and Google connectors:

• TAC for AWS — StrandsConnector, BedrockConnector, BedrockAgentCoreConnector for AWS Strands, Bedrock Agents, and Bedrock AgentCore
• TAC for Microsoft — AgentFrameworkConnector and VoiceLiveConnector for Microsoft Agent Framework, Azure AI Foundry (including Voice Live), and Azure OpenAI
• TAC for Google — ADKAgentEngineConnector, StudioAgentEngineConnector, CXAgentStudioConnector, ConversationalAgentsConnector for Google ADK, Agent Studio, CX Agent Studio, and Conversational Agents (Dialogflow CX)

Documentation:

• CLAUDE.md - Architecture, development guide, and API reference
• Getting Started Guide - Setup instructions, environment variables, and troubleshooting

---

TAC Development / Contribution

TAC uses npm workspaces for package management. Ensure you have Node.js and npm installed:

node --version  # Should be 22.13.0 or newer
npm --version   # Should be 9 or newer

Setup Development Environment

# Clone repository
git clone https://github.com/twilio/twilio-agent-connect-typescript.git
cd twilio-agent-connect-typescript

# Install all dependencies
npm install

# Build all packages
npm run build

Running Tests and Checks

# Format code
npm run format

# Run linting
npm run lint

# Run type checking
npm run typecheck

# Run tests
npm test

# Run tests in watch mode (for development)
npm run test:watch

# Run all checks at once
npm run build && npm run lint && npm run typecheck && npm test