Show HN：Ably AI Transport – 為代理應用程式提供的傳輸層

About AI Transport

Ably AI Transport is a drop-in infrastructure layer that upgrades your AI streams into bi-directional, stateful experiences. It enables you to build multi-device, steerable AI applications that are agent agnostic, incredibly resilient and highly scalable.

AI Transport enables you to add a realtime delivery layer to your application, providing the infrastructure required to deliver modern, stateful AI experiences to users. It works seamlessly with any AI model or framework, such as OpenAI, Anthropic, Vercel or LangChain.

AI Transport runs on Ably's fault-tolerant and highly-available platform. The platform supports streaming data between all internet-connected devices at low latencies across the globe. Its elastic global infrastructure delivers enterprise-scale messaging that effortlessly scales to meet demand.

Drop AI Transport into your applications to transform them into modern, bi-directional AI experiences that keep users engaged. AI Transport provides the building blocks to deliver reliable, resumable token streams with robust session management and state hydration to always keep your users and agents in sync.

Get started

Start learning the basics of AI Transport right away with a getting started guide using your agent and framework of choice:

OpenAI

Use the following guides to get started with OpenAI:

Message-per-response

Stream OpenAI responses using message appends

Message-per-token

Stream OpenAI responses using individual token messages

Anthropic

Use the following guides to get started with Anthropic:

Message-per-response

Stream Anthropic responses using message appends

Message-per-token

Stream Anthropic responses using individual token messages

Features

AI Transport provides a range of features built on Ably's highly-scalable realtime platform to enable you to deliver reliable, stateful AI experiences that provide the first-class UX your users expect from modern applications.

Token streaming

Token streaming is the core of how LLMs deliver their responses to users. Tokens are progressively streamed to users from your LLM so that users don't need to wait for a complete response before seeing any output.

Using AI Transport, your token streams are reliable and persistent. They survive modern environments where users change browser tabs, refresh the page or switch devices, and common interruptions such as temporary network loss. Your users can always reconnect and continue where they left off without having to start over.

Read more about token streaming.

Bi-directional communication

AI Transport supports rich, bi-directional communication patterns between users and agents.

Build sophisticated AI experiences with features like accepting user input for interactive conversations, streaming chain-of-thought reasoning for transparency, attaching citations to responses for verifiability, implementing human-in-the-loop workflows for sensitive operations, and exposing tool calls for generative UI and visibility.

These messaging features work seamlessly with token streaming to create complete, interactive AI experiences.

Read more about messaging features.

Durable sessions

AI Transport enables durable sessions that persist beyond the lifetime of individual connections, allowing users and agents to connect and disconnect independently.

Communication shouldn't be tied to the connection state of either party. If a user goes offline or their connection drops, they should be able to continue their session without losing context. AI Transport provides robust session management by enabling users and agents to connect independently of one another.

Your users can start a conversation on their mobile and seamlessly continue it on their desktop. Similarly, multiple users can participate in the same conversation with a single agent and they will all remain in sync, in realtime.

Read more about sessions and identity.

Automatic catch-up

AI Transport enables clients to hydrate conversation and session state from the channel, including message history and in-progress responses.

Whether a user is briefly disconnected when they drive through a tunnel, or they're rejoining a conversation the following day of work, AI Transport allows clients to resynchronise the full conversation state, including both historical messages and in-progress responses. Your users are always up to date with the full conversation, in order, anywhere.

Read more about client hydration.

Background processing

AI Transport allows agents to process jobs in the background while users go offline, with full awareness of their online status through realtime presence tracking.

Users can work asynchronously by prompting an agent to perform a task without having to monitor its progress. They can go offline and receive a push notification when the agent has completed the task, or reconnect at any time to seamlessly resume and see all progress made while they were away using state hydration.

It also puts you in control of how you manage your application when there aren't any users online. For example, you can choose whether to pause a conversation when a user exits their browser tab, or allow the agent to complete its response for the user to view when they return.

Read more about status-aware cost controls.

Enterprise controls

Ably's platform provides integrations and functionality to ensure that your applications always exceed the requirements of enterprise environments. Whether that's message auditing, client identification or fine-grained authorization.

Examples

Take a look at some example code running in-browser of the sorts of features you can build with AI Transport underpinning your applications:

Message per response streaming

Stream individual tokens from AI models into a single message.

Message per response streaming

Stream individual tokens from AI models into a single message.

Message per token streaming

Stream individual tokens from AI models as separate messages.

Message per token streaming

Stream individual tokens from AI models as separate messages.

Pricing

AI Transport uses Ably's usage based billing model at your package rates. Your consumption costs will depend on the number of messages inbound (published to Ably) and outbound (delivered to subscribers), and how long channels or connections are active. Contact Ably to discuss options for Enterprise pricing and volume discounts.

The cost of streaming token responses over Ably depends on:

For example, suppose an AI support chatbot sends a response of 300 tokens, each as a discrete update, using the message-per-response pattern, and with a single client subscribed to the channel. With AI Transport's append rollup,those 300 input tokens will be conflated to 100 discrete inbound messages, resulting in 100 outbound messages and 100 persisted messages. See the AI support chatbot pricing example for a full breakdown of the costs in this scenario.