generateWorld

[tombeckenham]

Summary

Add a generateWorld() activity for live, prompt-steerable world models, plus a first-party Reactor adapter. The example model is Visko Orbis (visko-orbis-stable / visko-orbis-dynamic) on Reactor.

Problem

TanStack AI already has generateImage, generateVideo, generateAudio, and generateSpeech. Those are one-shot jobs. World models are different: you open a session, stream video, and steer with new prompts while the stream runs.

Reactor hosts those models (Orbis, Happy Oyster, LingBot World, Helios) behind one SDK. There is no TanStack AI activity or adapter for that shape.

Scope

1. generateWorld() in @tanstack/ai (kind: 'world'), same middleware / otel / abort / stream envelope as the other media activities.
2. @tanstack/ai-reactor first-party adapter. Server mints a session-scoped JWT (POST https://api.reactor.inc/tokens). The browser connects with @reactor-team/js-sdk, sets the prompt, and starts the stream.
3. Example app examples/ts-react-world — server mints the token, browser plays and steers the stream.
4. Docs for the activity and the Reactor adapter.

Example

import { generateWorld } from '@tanstack/ai'
import { reactorWorld } from '@tanstack/ai-reactor'

const world = await generateWorld({
  adapter: reactorWorld('visko-orbis-stable'),
  prompt:
    'A dramatic coastline of black volcanic cliffs at golden hour, a single unbroken take.',
})

// world.token, world.model, world.prompt — hand these to the browser

The browser connects with the token, calls set_prompt, then start. Prompt changes mid-run morph the scene at the next chunk (~1.8s on Orbis).

Google / Gemini

Looked at adding this to @tanstack/ai-gemini. Google DeepMind Genie 3 / Project Genie is Labs-only (AI Ultra). There is no public Gemini API or Vertex method for world generation as of 2026-09-04. Do not stub geminiWorld. The WorldAdapter interface is the extension point when Google ships an API.

Non-goals

• Client React hook (useGenerateWorld) — follow-up
• WASD / look controls (Happy Oyster, LingBot) beyond the shared prompt/start/pause surface
• A Python Reactor Runtime plugin
• Shipping API keys to the browser

Acceptance

• generateWorld() in @tanstack/ai with tests and otel mapping (world_generation)
• @tanstack/ai-reactor mints scoped tokens for Orbis (and other Reactor world models)
• Example app examples/ts-react-world
• E2E covers the activity
• Docs: world generation guide + Reactor adapter page
• Changeset