feat: add @cloudflare/tanstack-ai package

[vaibhavshn]

Summary

Adds @cloudflare/tanstack-ai — the canonical package for using TanStack AI with Cloudflare. Supports Workers AI (direct binding and REST API), third-party providers (OpenAI, Anthropic, Gemini, Grok) routed through AI Gateway, and plain Workers AI without a gateway.

Also includes a full-featured demo app at demos/tanstack-ai-chat-react.

Package: @cloudflare/tanstack-ai

Adapters

Capability	OpenAI	Anthropic	Gemini	Grok	Workers AI
Chat	createOpenAiChat	createAnthropicChat	createGeminiChat	createGrokChat	createWorkersAiChat
Image	createOpenAiImage	—	createGeminiImage	createGrokImage	—
Summarize	createOpenAiSummarize	createAnthropicSummarize	createGeminiSummarize	createGrokSummarize	—
Transcription	createOpenAiTranscription	—	—	—	—
TTS	createOpenAiTts	—	—	—	—
Video	createOpenAiVideo	—	—	—	—

Workers AI config modes

// Direct binding — zero config, no API keys
createWorkersAiChat(model, { binding: env.AI })

// Direct REST — no binding needed
createWorkersAiChat(model, { accountId, apiKey })

// AI Gateway (binding) — routed through gateway
createWorkersAiChat(model, { binding: env.AI.gateway(id), apiKey })

// AI Gateway (credentials) — fully credential-based
createWorkersAiChat(model, { gatewayId, accountId, cfApiKey })

Design decisions

openai as a hard dependency

The Workers AI adapter internally uses the OpenAI SDK to provide a consistent interface. In direct binding mode (env.AI), a custom fetch shim intercepts OpenAI SDK requests, extracts the model/params from the JSON body, calls env.AI.run(), and transforms the Workers AI response into OpenAI-compatible SSE chunks via a TransformStream. This means openai is always needed at runtime, not just for the OpenAI provider.

Structural discrimination for binding vs gateway

env.AI and env.AI.gateway(id) are both objects with a .run() method, but env.AI also has a .gateway() method. We use this to distinguish them at runtime (isDirectBindingConfig checks for the presence of .gateway on the binding), avoiding any need for explicit mode or type config fields.

Gemini: credentials only, no binding support

The @google/genai SDK doesn't support a custom fetch override — only baseUrl and headers via httpOptions. So the Gemini adapter can't use the AI Gateway binding (env.AI.gateway(id).run()). Instead it directly sets baseUrl to the AI Gateway REST endpoint (gateway.ai.cloudflare.com/v1/{accountId}/{gatewayId}/google-ai-studio). A runtime guard throws a clear error if a binding config is accidentally passed. Tracking upstream: googleapis/js-genai#999.

Provider SDKs as optional dependencies

@tanstack/ai-openai, @tanstack/ai-anthropic, @tanstack/ai-gemini, @tanstack/ai-grok, and their underlying provider SDKs (@anthropic-ai/sdk, @google/genai) are all optionalDependencies. Users only install what they need. Each adapter file is a separate entry point in tsup and package.json exports, so tree-shaking works and unused adapters aren't bundled.

Workers AI embeddings/image generation held back

The adapters for WorkersAiEmbeddingAdapter and WorkersAiImageAdapter are fully implemented (workers-ai-embedding.ts, workers-ai-image.ts) but intentionally excluded from the public API and build output. TanStack AI doesn't yet export BaseEmbeddingAdapter or BaseImageAdapter for third-party use — once they do, these are ready to ship with no code changes.

Stable stream IDs

transformWorkersAiStream generates a single streamId and created timestamp at the start of each stream and reuses them across all SSE chunks, matching OpenAI's convention. Tool call IDs use a monotonic counter per stream (call_{streamId}_{n}).

Demo: demos/tanstack-ai-chat-react

A Cloudflare Workers + Vite + React demo with three tabs:

• Chat — streaming multi-provider chat with tool calling (Llama 4 Scout, Qwen3 30B, GPT-5.2, Claude Sonnet 4.5, Gemini 2.5 Flash, Grok 4)
• Images — image generation with OpenAI, Gemini, and Grok
• Summarize — text summarization with OpenAI, Anthropic, and Gemini

Users can enter their own Cloudflare credentials (Account ID, AI Gateway ID, API Token) directly in the frontend UI — stored in sessionStorage, passed as request headers. The worker reads credentials from headers and falls back to environment variables for deployed instances.

Tests

• Gateway fetch — createGatewayFetch with binding/credentials configs, cache headers, Workers AI provider specifics, endpoint extraction
• Binding fetch — createWorkersAiBindingFetch request translation, stream transform, stable chunk IDs, OpenAI-format passthrough
• Config detection — isDirectBindingConfig, isDirectCredentialsConfig, isGatewayConfig
• WorkersAiTextAdapter — chatStream and structuredOutput across all config modes, error emission
• Message builders — buildOpenAIMessages, buildOpenAITools
• Public API surface — verifies correct exports, confirms held-back adapters are not exported
• E2E (Workers AI REST) — 12 models tested against live API for chat, multi-turn, tool calling, tool round-trip, and structured output
• E2E (Workers AI Binding) — same 12 models tested via env.AI binding through a wrangler dev harness

Notes for reviewers

1. The as WorkersAiTextModel casts in the demo: the @cloudflare/workers-types conditional type that derives WorkersAiTextModel doesn't structurally match some model names (e.g. @cf/meta/llama-4-scout-17b-16e-instruct). The casts are only in the demo, not the library. Worth investigating if this is a workers-types bug.

2. Gemini limitation: the @google/genai SDK lacks a fetch override, so there's no way to use the AI binding for Gemini. The upstream issue is googleapis/js-genai#999 (has an open PR: #1215). We've added a runtime guard that throws a clear error if a binding config is passed.

3. openai version: pinned to ^6.16.0 as a hard dep. This is the minimum version that supports the APIs we need. If there are concerns about the dep size, we could consider vendoring just the streaming logic, but that's a lot of surface area.

4. Workers AI embedding/image adapters: the code is complete in workers-ai-embedding.ts and workers-ai-image.ts but excluded from tsup.config.ts entry points and index.ts exports. Once TanStack AI ships BaseEmbeddingAdapter/BaseImageAdapter, we flip the switch — no new code needed.

5. Demo credentials UX: credentials entered in the frontend are passed as X-CF-Account-Id, X-CF-Gateway-Id, X-CF-Api-Token headers. The worker checks for these first, then falls back to env vars. This means the deployed demo works both with and without server-side secrets configured.

[changeset-bot]

🦋 Changeset detected

Latest commit: 95f8b48

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
workers-ai-provider	Minor
@cloudflare/tanstack-ai	Minor
@cloudflare/tanstack-ai-example	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

Open in StackBlitz

npx https://pkg.pr.new/cloudflare/ai/ai-gateway-provider@389

npx https://pkg.pr.new/cloudflare/ai/@cloudflare/tanstack-ai@389

npx https://pkg.pr.new/cloudflare/ai/workers-ai-provider@389

commit: 95f8b48

[threepointone]

Taking over this PR.

[threepointone]

Hey @vaibhavshn — thanks for the original work here, I've built on top of it quite a bit. Here's a summary of what changed:

Scope expansion: The package is now the canonical way to use TanStack AI with Cloudflare — not just AI Gateway, but also plain Workers AI (direct binding and REST API, no gateway required). Think of it as the TanStack AI equivalent of workers-ai-provider.

API changes:

• openai is now a hard dependency (the Workers AI adapter uses it internally for the binding shim)
• Config keys changed — e.g. ai: env.AI → binding: env.AI to match Cloudflare conventions
• Workers AI embedding and image adapters are written but held back from the public API until TanStack AI exports the base adapter types for those capabilities
• Provider SDKs remain optional dependencies — users only install what they need

Build & packaging:

• Aligned tsconfig.json, tsup.config.ts, and package.json exports with workers-ai-provider and ai-gateway-provider for consistency across the monorepo
• moduleResolution: "bundler", no file extensions on relative imports
• Added noUncheckedIndexedAccess, noFallthroughCasesInSwitch, noUnusedLocals, noUnusedParameters
• Renamed the directory from tanstack-ai-adapters to tanstack-ai

Tests: Added comprehensive test suites — gateway fetch, binding fetch, config detection, Workers AI adapter (all config modes), message builders, and public API surface verification. 80 tests total.

Demo app overhaul:

• Added Image Generation and Summarize tabs (using direct fetch since @tanstack/ai-react only has useChat)
• Added a frontend credentials UI — users can enter their Cloudflare Account ID, Gateway ID, and API Token directly in the browser (stored in localStorage, passed as headers). The worker falls back to env vars if no headers are present.
• Updated to latest models (GPT-5.2, Gemini 2.5 Flash, Grok 4, Llama 4 Scout 17B, Qwen3 30B)
• Added plain Workers AI as a chat provider option (no gateway)
• General style cleanup

Important — I need you to test this end to end. I don't have API keys set up, so I haven't been able to actually run the demo or verify the adapters against live services. Please test all the providers and config modes, especially:

• Workers AI via binding (env.AI) and via REST API
• Workers AI through AI Gateway (both binding and credentials modes)
• The third-party providers (OpenAI, Anthropic, Gemini, Grok) through AI Gateway
• The demo app — all three tabs, both with env vars and with frontend-entered credentials
• The Gemini adapter specifically, since it can't use the binding path (only credentials)

Review the code too and flag anything that looks off.

[vaibhavshn]

There is an already existing issue in Google GenAI repo for supporting custom fetch implementation. Have sent a message and linked to it above.

[threepointone]

Ok I pushed a whole bunch of changes. Importantly, I've fixed al the quirks around workers ai, and the demo is a lot more comprehensive now. feel free to looks at individual commits I've pushed since my last comment. I also rewrote the PR description. I think this is ready to review and land once you've tested e2e.

[threepointone]

ok pushed, let's see what CI says

[threepointone]

CI passes! please test end to end, the demo app should be pretty comprehensive