Documentation
¶
Overview ¶
Package runtime provides a provider-agnostic AI runtime for the ai-sdk.
It glues together the domain interfaces (chat, embed, etc.), provider implementations, and the models.dev catalog so that applications can resolve a model reference such as "openai/gpt-5.4" into a working chat provider without hardcoding every provider themselves.
The runtime is intentionally layered above core: it imports domain interfaces and provider implementations, then delegates the actual chat/embed orchestration to core.GenerateText / core.StreamText.
Key abstractions:
- ProviderClass: a pluggable factory that turns a ProviderConfig into a ProviderSet of domain implementations. Built-in classes include "openai-compatible", which uses provider/openai with arbitrary base URLs.
- Catalog: the in-memory view of the models.dev provider/model metadata, loaded from a network snapshot, an embedded fallback, or caller-supplied JSON.
- Runtime: the public entry point. It resolves model references to provider instances, caches them, and exposes Chat/ChatStream calls.
Applications can add custom provider classes by calling RegisterClass before constructing a Runtime, or by populating ProviderConfig entries with a known class name.
Package runtime provides built-in OAuth PKCE authentication for providers.
This implementation is written from scratch using only the Go standard library. It performs a local-callback PKCE flow and caches the resulting access token on disk. Callers must supply OpenBrowser if they want the system browser opened automatically; otherwise the authorization URL is printed to stderr.
Index ¶
- Constants
- Variables
- func ClassNames() []string
- func ClearClasses()
- func DefaultBaseURL(class string) (string, bool)
- func RegisterAuthResolver(t AuthType, r AuthResolver)
- func RegisterBuiltinClasses()
- func RegisterClass(c ProviderClass)
- func ResolveAPIKey(cfg ProviderConfig) (string, error)
- type AuthConfig
- type AuthResolver
- type AuthResolverFunc
- type AuthResult
- type AuthType
- type Capability
- type Catalog
- type CatalogModel
- type CatalogOptions
- type CatalogProvider
- type Config
- type CostConfig
- type ModelConfig
- type ModelInfo
- type ModelRef
- type OAuthPKCEResolver
- type ProviderClass
- type ProviderConfig
- type ProviderSet
- type ReasoningOption
- type Runtime
- func (r *Runtime) Catalog() *Catalog
- func (r *Runtime) Chat(ctx context.Context, ref string, opts core.GenerateOptions) (core.GenerateResult, error)
- func (r *Runtime) ChatModelInfo(ref string) (chat.ModelInfo, error)
- func (r *Runtime) ChatProvider(ctx context.Context, ref string) (chat.Provider, string, error)
- func (r *Runtime) ChatStream(ctx context.Context, ref string, opts core.GenerateOptions) (core.StreamResult, error)
- func (r *Runtime) Decide(ctx context.Context, ref string, req decide.Request) (decide.Response, error)
- func (r *Runtime) DecideProvider(ctx context.Context, ref string) (decide.Provider, string, error)
- func (r *Runtime) Image(ctx context.Context, ref string, req image.GenerateImageRequest) (image.GenerateImageResponse, error)
- func (r *Runtime) ImageProvider(ctx context.Context, ref string) (image.Provider, string, error)
- func (r *Runtime) LoadCatalog(ctx context.Context) error
- func (r *Runtime) Models(providerID string) ([]ModelInfo, error)
- func (r *Runtime) Music(ctx context.Context, ref string, req music.GenerateMusicRequest) (music.GenerateMusicResponse, error)
- func (r *Runtime) MusicProvider(ctx context.Context, ref string) (music.Provider, string, error)
- func (r *Runtime) Object(ctx context.Context, ref string, req object.Request) (object.ObjectResult, error)
- func (r *Runtime) ObjectProvider(ctx context.Context, ref string) (object.Provider, string, error)
- func (r *Runtime) ParseModelRef(ref string) (ModelRef, error)
- func (r *Runtime) Rerank(ctx context.Context, ref string, req rerank.Request) (rerank.Response, error)
- func (r *Runtime) RerankProvider(ctx context.Context, ref string) (rerank.Provider, string, error)
- func (r *Runtime) ResolveAuth(ctx context.Context, providerID string) (AuthResult, error)
- func (r *Runtime) Speech(ctx context.Context, ref string, req speech.GenerateSpeechRequest) (speech.GenerateSpeechResponse, error)
- func (r *Runtime) SpeechProvider(ctx context.Context, ref string) (speech.Provider, string, error)
- func (r *Runtime) SpeechStream(ctx context.Context, ref string, req speech.GenerateSpeechRequest) (speech.SpeechStream, error)
- func (r *Runtime) SpeechStreamProvider(ctx context.Context, ref string) (speech.Streamer, string, error)
- func (r *Runtime) Transcribe(ctx context.Context, ref string, req transcribe.TranscribeRequest) (transcribe.TranscribeResponse, error)
- func (r *Runtime) TranscribeProvider(ctx context.Context, ref string) (transcribe.Provider, string, error)
- func (r *Runtime) Video(ctx context.Context, ref string, req video.GenerateVideoRequest) (video.GenerateVideoResponse, error)
- func (r *Runtime) VideoProvider(ctx context.Context, ref string) (video.Provider, string, error)
Constants ¶
const DefaultCatalogURL = catalog.DefaultURL
DefaultCatalogURL is the public models.dev provider API endpoint.
Variables ¶
var ErrCapabilityNotSupported = fmt.Errorf("runtime: capability not supported")
ErrCapabilityNotSupported is returned when the resolved provider cannot satisfy the requested capability.
ErrCatalogUnavailable is returned when the catalog cannot be loaded from any source.
var ErrClassNotFound = fmt.Errorf("runtime: provider class not found")
ErrClassNotFound is returned when a provider's class is not registered.
var ErrProviderNotFound = fmt.Errorf("runtime: provider not found")
ErrProviderNotFound is returned when a model reference cannot be mapped to a configured or catalog provider.
var NPMClassMapping = catalog.NPMClassMapping
NPMClassMapping maps models.dev npm package identifiers to the provider class names registered by RegisterBuiltinClasses.
Functions ¶
func ClassNames ¶
func ClassNames() []string
ClassNames returns the names of all registered classes in sorted order.
func ClearClasses ¶
func ClearClasses()
ClearClasses removes all registered classes. It is intended for tests.
func DefaultBaseURL ¶ added in v0.7.0
DefaultBaseURL returns the base URL a built-in provider class uses when its ProviderConfig sets no BaseURL. The second return value is false for classes without a default and for unregistered class names.
func RegisterAuthResolver ¶
func RegisterAuthResolver(t AuthType, r AuthResolver)
RegisterAuthResolver registers a resolver for an auth type. It overwrites any existing resolver for that type.
func RegisterBuiltinClasses ¶
func RegisterBuiltinClasses()
RegisterBuiltinClasses registers the provider classes and auth resolvers shipped with the ai-sdk. Call this once at program startup before constructing a Runtime.
func RegisterClass ¶
func RegisterClass(c ProviderClass)
RegisterClass registers a provider class. Calling RegisterClass with the same name twice is a no-op (the first registration wins).
func ResolveAPIKey ¶
func ResolveAPIKey(cfg ProviderConfig) (string, error)
ResolveAPIKey extracts an API key from ProviderConfig, preferring the environment variable named by Auth.APIKeyEnv, then a literal APIKey.
Types ¶
type AuthConfig ¶
type AuthConfig struct {
Type AuthType `json:"type,omitempty"`
APIKeyEnv string `json:"api_key_env,omitempty"`
APIKey string `json:"api_key,omitempty"`
AuthorizeURL string `json:"authorize_url,omitempty"`
TokenURL string `json:"token_url,omitempty"`
ClientID string `json:"client_id,omitempty"`
IDP string `json:"idp,omitempty"`
TokenAuthMethod string `json:"token_auth_method,omitempty"`
}
AuthConfig describes how to obtain a bearer token or API key for a provider. Not all fields are used by every AuthType.
type AuthResolver ¶
type AuthResolver interface {
// Resolve returns a credential for the given provider configuration.
Resolve(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
}
AuthResolver resolves an AuthConfig into a credential at call time. Custom resolvers can be registered per class to handle bespoke auth flows (OAuth device code, short-lived token exchange, etc.).
func GetAuthResolver ¶
func GetAuthResolver(t AuthType) (AuthResolver, bool)
GetAuthResolver returns the registered resolver for t, or false if none exists.
type AuthResolverFunc ¶
type AuthResolverFunc func(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
AuthResolverFunc adapts a function to the AuthResolver interface.
func (AuthResolverFunc) Resolve ¶
func (f AuthResolverFunc) Resolve(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
Resolve implements AuthResolver.
type AuthResult ¶
AuthResult is the resolved credential for a provider. It may be empty for providers that require no authentication.
func (AuthResult) IsEmpty ¶
func (r AuthResult) IsEmpty() bool
IsEmpty reports whether the result carries no token or headers.
type Capability ¶
type Capability string
Capability names a model capability a ProviderClass may satisfy.
const ( CapabilityChat Capability = "chat" CapabilityEmbed Capability = "embed" CapabilityImage Capability = "image" CapabilityVideo Capability = "video" CapabilityObject Capability = "object" CapabilityDecide Capability = "decide" CapabilityRerank Capability = "rerank" CapabilitySpeech Capability = "speech" CapabilityTranscribe Capability = "transcribe" CapabilityMusic Capability = "music" )
Standard capabilities.
type Catalog ¶
The models.dev catalog lives in package catalog; these names keep the runtime's API.
func NewCatalog ¶
func NewCatalog(opts CatalogOptions) *Catalog
NewCatalog returns an empty catalog configured by opts.
type CatalogModel ¶
The models.dev catalog lives in package catalog; these names keep the runtime's API.
type CatalogOptions ¶
The models.dev catalog lives in package catalog; these names keep the runtime's API.
type CatalogProvider ¶
The models.dev catalog lives in package catalog; these names keep the runtime's API.
type Config ¶
type Config struct {
// DefaultProvider is the provider ID used when a model reference
// does not include a provider prefix.
DefaultProvider string `json:"default_provider,omitempty"`
// Providers are the configured provider instances. The map key is
// the local provider ID and must match ProviderConfig.ID.
Providers map[string]ProviderConfig `json:"providers,omitempty"`
// CatalogURL overrides the default models.dev endpoint. Empty means
// use DefaultCatalogURL.
CatalogURL string `json:"catalog_url,omitempty"`
// CatalogCachePath enables on-disk caching of the fetched catalog.
CatalogCachePath string `json:"catalog_cache_path,omitempty"`
// HTTPTransport, when set, carries every provider request (for example
// an otelhttp-wrapped transport). A ProviderConfig.Transport takes
// precedence. Nil uses http.DefaultTransport.
HTTPTransport http.RoundTripper `json:"-"`
}
Config is the declarative runtime configuration. Applications supply this (e.g. from JSON/YAML) and the Runtime turns it into working provider instances.
func (Config) ProviderByID ¶
func (c Config) ProviderByID(id string) (ProviderConfig, bool)
ProviderByID returns the provider config with the given id, or false.
type CostConfig ¶
type CostConfig struct {
Input float64 `json:"input,omitzero"`
Output float64 `json:"output,omitzero"`
CacheRead float64 `json:"cache_read,omitzero"`
CacheWrite float64 `json:"cache_write,omitzero"`
}
CostConfig carries per-token pricing metadata.
type ModelConfig ¶
type ModelConfig struct {
ID string `json:"id"`
Name string `json:"name,omitempty"`
URL string `json:"url,omitempty"`
ContextWindow int `json:"context_window,omitzero"`
MaxOutputTokens int `json:"max_output_tokens,omitzero"`
Reasoning bool `json:"reasoning,omitzero"`
ToolCall bool `json:"tool_call,omitzero"`
StructuredOutput bool `json:"structured_output,omitzero"`
// Temperature reports whether the model accepts a temperature parameter.
// nil means unspecified (unknown), true/false override catalog metadata
// explicitly.
Temperature *bool `json:"temperature,omitzero"`
// Capabilities optionally declares the model's capabilities, overriding
// metadata-derived ones. When non-empty it is authoritative: a capability
// absent from the list is treated as unsupported even when catalog
// modalities would imply it.
Capabilities []Capability `json:"capabilities,omitempty"`
Cost CostConfig `json:"cost"`
Extra map[string]any `json:"extra,omitempty"`
}
ModelConfig is a configured model entry for a provider. It is merged with catalog metadata by the runtime.
type ModelInfo ¶
type ModelInfo struct {
ID string
ProviderID string
Name string
URL string
ContextWindow int
MaxOutputTokens int
Reasoning bool
ReasoningOptions []ReasoningOption
ToolCall bool
StructuredOutput bool
// Temperature reports whether the model accepts a temperature parameter.
// nil means unknown (the catalog published no flag and no override set
// one), false means the model rejects it.
Temperature *bool
// InputModalities and OutputModalities mirror the provider's published
// modalities (for example models.dev's audio/speech, text, image, video).
// Empty means unknown.
InputModalities []string
OutputModalities []string
// Capabilities, when non-empty, is the authoritative capability list and
// overrides the modality-derived inference.
Capabilities []Capability
Cost CostConfig
Extra map[string]any
}
ModelInfo is the runtime's normalised view of a model, combining catalog data and configured overrides.
type ModelRef ¶
ModelRef is a parsed model reference of the form "provider/model" or just "model" when a default provider is configured.
type OAuthPKCEResolver ¶
type OAuthPKCEResolver struct {
// CacheDir overrides the directory used to store access tokens.
// Defaults to os.UserConfigDir()/ai-sdk/oauth.
CacheDir string
// OpenBrowser wraps browser.OpenURL. Tests can override it.
OpenBrowser func(url string) error
}
OAuthPKCEResolver is the built-in AuthResolver for AuthTypeOAuthPKCE. It is registered by RegisterBuiltinClasses so that providers configured with auth.type = "oauth_pkce" can obtain bearer tokens automatically.
func (*OAuthPKCEResolver) Resolve ¶
func (r *OAuthPKCEResolver) Resolve(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
Resolve implements AuthResolver.
type ProviderClass ¶
type ProviderClass interface {
// Name returns the stable class identifier used in ProviderConfig.Class.
Name() string
// Supports reports whether this class can satisfy cap.
Supports(cap Capability) bool
// New builds a ProviderSet from cfg for the given model. ctx may be
// used for short-lived setup requests (e.g. discovery or token
// exchange), but must not be stored.
New(ctx context.Context, cfg ProviderConfig, model ModelInfo) (ProviderSet, error)
}
ProviderClass is a factory for provider instances. Each class knows how to turn a ProviderConfig (base URL, auth, options) and a resolved model into one or more domain providers.
func GetClass ¶
func GetClass(name string) (ProviderClass, bool)
GetClass returns a registered class by name.
func MustRegisterClass ¶
func MustRegisterClass(c ProviderClass) ProviderClass
MustRegisterClass is like RegisterClass but returns the class so it can be chained in var initializations.
type ProviderConfig ¶
type ProviderConfig struct {
// ID is the local identifier for this provider instance (e.g. "openai",
// "my-maas"). It must be unique within a Runtime.
ID string
// Class selects the registered ProviderClass factory. Examples:
// "openai-compatible", "anthropic-messages".
Class string
// BaseURL is the provider's API base URL. Individual models may
// override this via their own URL field.
BaseURL string
// Auth describes how to obtain credentials for this provider.
Auth AuthConfig
// Headers are extra HTTP headers merged into every request.
Headers map[string]string
// Insecure disables TLS certificate verification.
Insecure bool
// Timeout is an optional whole-request timeout in milliseconds. Values
// greater than zero opt in to a timeout; zero leaves requests unlimited so
// callers can control cancellation through context deadlines.
Timeout int
// Transport carries this provider's HTTP requests. Nil falls back to
// Config.HTTPTransport, then http.DefaultTransport.
Transport http.RoundTripper `json:"-"`
// Options carries class-specific options as a generic map. Classes
// read only the keys they understand.
Options map[string]any
// Models allows per-provider model overrides/advertisements. These
// augment or override the catalog when the runtime resolves model
// references.
Models []ModelConfig
}
ProviderConfig is the minimal information needed to construct a provider instance. The Class field selects which ProviderClass factory is invoked.
type ProviderSet ¶
type ProviderSet struct {
Chat chat.Provider
Embed embed.Provider
Image image.Provider
Video video.Provider
Object object.Provider
Decide decide.Provider
Rerank rerank.Provider
Speech speech.Provider
Transcribe transcribe.Provider
Music music.Provider
}
ProviderSet is a collection of domain providers produced by a single ProviderClass instance.
func (ProviderSet) Has ¶
func (s ProviderSet) Has(cap Capability) bool
Has reports whether the set satisfies cap.
type ReasoningOption ¶
type ReasoningOption = catalog.ReasoningOption
The models.dev catalog lives in package catalog; these names keep the runtime's API.
type Runtime ¶
type Runtime struct {
// contains filtered or unexported fields
}
Runtime resolves model references to provider instances and exposes a high-level chat/embed API. It caches provider instances so repeated calls to the same provider/model/URL reuse the same underlying HTTP client.
func NewRuntime ¶
NewRuntime creates a Runtime from the supplied configuration. It does not load the catalog automatically; call LoadCatalog (or supply a pre-populated Catalog) before making calls.
func NewRuntimeWithCatalog ¶
NewRuntimeWithCatalog creates a Runtime that uses an already-populated catalog.
func (*Runtime) Catalog ¶
Catalog returns the runtime's catalog. It may be nil until LoadCatalog is called.
func (*Runtime) Chat ¶
func (r *Runtime) Chat(ctx context.Context, ref string, opts core.GenerateOptions) (core.GenerateResult, error)
Chat performs a non-streaming chat completion for the given model reference. The model field inside opts is overwritten with the resolved model ID.
func (*Runtime) ChatModelInfo ¶ added in v0.2.0
ChatModelInfo resolves a model reference to the request-shaping flags that belong on a chat request, without building the provider. Callers that resolve their own provider (for example agentloop) use it so their requests carry the same shaping flags as Chat and ChatStream.
func (*Runtime) ChatProvider ¶
ChatProvider resolves a model reference to a chat.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.
func (*Runtime) ChatStream ¶
func (r *Runtime) ChatStream(ctx context.Context, ref string, opts core.GenerateOptions) (core.StreamResult, error)
ChatStream performs a streaming chat completion for the given model reference.
func (*Runtime) Decide ¶ added in v0.2.0
func (r *Runtime) Decide(ctx context.Context, ref string, req decide.Request) (decide.Response, error)
Decide answers named questions using the resolved model.
func (*Runtime) DecideProvider ¶ added in v0.2.0
DecideProvider resolves a decision provider, refusing published metadata that excludes decisions. Unknown compatible models are discovered from the server.
func (*Runtime) Image ¶ added in v0.1.31
func (r *Runtime) Image(ctx context.Context, ref string, req image.GenerateImageRequest) (image.GenerateImageResponse, error)
Image performs a non-streaming image generation for the given model reference. The model field inside req is overwritten with the resolved model ID.
func (*Runtime) ImageProvider ¶ added in v0.1.31
ImageProvider resolves a model reference to an image.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.
func (*Runtime) LoadCatalog ¶
LoadCatalog fetches or loads the models.dev catalog according to the runtime configuration. If cfg.CatalogURL or cfg.CatalogCachePath are set, they override the catalog defaults.
func (*Runtime) Models ¶
Models returns the resolved model information for a provider, merged from configured overrides and the catalog.
func (*Runtime) Music ¶ added in v0.1.38
func (r *Runtime) Music(ctx context.Context, ref string, req music.GenerateMusicRequest) (music.GenerateMusicResponse, error)
Music generates a music track for the given model reference. The model field inside req is overwritten with the resolved model ID.
func (*Runtime) MusicProvider ¶ added in v0.1.38
MusicProvider resolves a model reference to a music.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.
func (*Runtime) Object ¶ added in v0.1.31
func (r *Runtime) Object(ctx context.Context, ref string, req object.Request) (object.ObjectResult, error)
Object performs a non-streaming object generation for the given model reference. The model field inside req is overwritten with the resolved model ID.
func (*Runtime) ObjectProvider ¶ added in v0.1.31
ObjectProvider resolves a model reference to an object.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.
func (*Runtime) ParseModelRef ¶
ParseModelRef splits a model reference into provider and model IDs. If the reference has no "/" separator and a default provider is configured, the default provider is used.
func (*Runtime) Rerank ¶ added in v0.1.31
func (r *Runtime) Rerank(ctx context.Context, ref string, req rerank.Request) (rerank.Response, error)
Rerank re-orders documents for the given model reference. The model field inside req is overwritten with the resolved model ID.
func (*Runtime) RerankProvider ¶ added in v0.1.31
RerankProvider resolves a model reference to a rerank.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.
func (*Runtime) ResolveAuth ¶
ResolveAuth resolves credentials for a configured provider using its registered auth resolver.
func (*Runtime) Speech ¶ added in v0.1.29
func (r *Runtime) Speech(ctx context.Context, ref string, req speech.GenerateSpeechRequest) (speech.GenerateSpeechResponse, error)
Speech performs a non-streaming text-to-speech generation for the given model reference. The model field inside req is overwritten with the resolved model ID.
func (*Runtime) SpeechProvider ¶ added in v0.1.29
SpeechProvider resolves a model reference to a speech.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.
func (*Runtime) SpeechStream ¶ added in v0.1.39
func (r *Runtime) SpeechStream(ctx context.Context, ref string, req speech.GenerateSpeechRequest) (speech.SpeechStream, error)
SpeechStream starts a streaming text-to-speech generation for the given model reference. The model field inside req is overwritten with the resolved model ID, and the caller must Close the returned stream when finished.
func (*Runtime) SpeechStreamProvider ¶ added in v0.1.39
func (r *Runtime) SpeechStreamProvider(ctx context.Context, ref string) (speech.Streamer, string, error)
SpeechStreamProvider resolves a model reference to a speech.Streamer. It returns ErrCapabilityNotSupported when the resolved provider cannot stream.
func (*Runtime) Transcribe ¶ added in v0.1.29
func (r *Runtime) Transcribe(ctx context.Context, ref string, req transcribe.TranscribeRequest) (transcribe.TranscribeResponse, error)
Transcribe performs a non-streaming audio transcription for the given model reference. The model field inside req is overwritten with the resolved model ID.
func (*Runtime) TranscribeProvider ¶ added in v0.1.29
func (r *Runtime) TranscribeProvider(ctx context.Context, ref string) (transcribe.Provider, string, error)
TranscribeProvider resolves a model reference to a transcribe.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.
func (*Runtime) Video ¶ added in v0.1.31
func (r *Runtime) Video(ctx context.Context, ref string, req video.GenerateVideoRequest) (video.GenerateVideoResponse, error)
Video performs a non-streaming video generation for the given model reference. The model field inside req is overwritten with the resolved model ID.
func (*Runtime) VideoProvider ¶ added in v0.1.31
VideoProvider resolves a model reference to a video.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.