runtime

package
v0.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: Apache-2.0 Imports: 48 Imported by: 0

Documentation

Overview

Package runtime provides a provider-agnostic AI runtime for the ai-sdk.

It glues together the domain interfaces (chat, embed, etc.), provider implementations, and the models.dev catalog so that applications can resolve a model reference such as "openai/gpt-5.4" into a working chat provider without hardcoding every provider themselves.

The runtime is intentionally layered above core: it imports domain interfaces and provider implementations, then delegates the actual chat/embed orchestration to core.GenerateText / core.StreamText.

Key abstractions:

  • ProviderClass: a pluggable factory that turns a ProviderConfig into a ProviderSet of domain implementations. Built-in classes include "openai-compatible", which uses provider/openai with arbitrary base URLs.
  • Catalog: the in-memory view of the models.dev provider/model metadata, loaded from a network snapshot, an embedded fallback, or caller-supplied JSON.
  • Runtime: the public entry point. It resolves model references to provider instances, caches them, and exposes Chat/ChatStream calls.

Applications can add custom provider classes by calling RegisterClass before constructing a Runtime, or by populating ProviderConfig entries with a known class name.

Package runtime provides built-in OAuth PKCE authentication for providers.

This implementation is written from scratch using only the Go standard library. It performs a local-callback PKCE flow and caches the resulting access token on disk. Callers must supply OpenBrowser if they want the system browser opened automatically; otherwise the authorization URL is printed to stderr.

Index

Constants

View Source
const DefaultCatalogURL = catalog.DefaultURL

DefaultCatalogURL is the public models.dev provider API endpoint.

Variables

View Source
var ErrCapabilityNotSupported = fmt.Errorf("runtime: capability not supported")

ErrCapabilityNotSupported is returned when the resolved provider cannot satisfy the requested capability.

View Source
var ErrCatalogUnavailable = catalog.ErrUnavailable

ErrCatalogUnavailable is returned when the catalog cannot be loaded from any source.

View Source
var ErrClassNotFound = fmt.Errorf("runtime: provider class not found")

ErrClassNotFound is returned when a provider's class is not registered.

View Source
var ErrProviderNotFound = fmt.Errorf("runtime: provider not found")

ErrProviderNotFound is returned when a model reference cannot be mapped to a configured or catalog provider.

View Source
var NPMClassMapping = catalog.NPMClassMapping

NPMClassMapping maps models.dev npm package identifiers to the provider class names registered by RegisterBuiltinClasses.

Functions

func ClassNames

func ClassNames() []string

ClassNames returns the names of all registered classes in sorted order.

func ClearClasses

func ClearClasses()

ClearClasses removes all registered classes. It is intended for tests.

func DefaultBaseURL added in v0.7.0

func DefaultBaseURL(class string) (string, bool)

DefaultBaseURL returns the base URL a built-in provider class uses when its ProviderConfig sets no BaseURL. The second return value is false for classes without a default and for unregistered class names.

func RegisterAuthResolver

func RegisterAuthResolver(t AuthType, r AuthResolver)

RegisterAuthResolver registers a resolver for an auth type. It overwrites any existing resolver for that type.

func RegisterBuiltinClasses

func RegisterBuiltinClasses()

RegisterBuiltinClasses registers the provider classes and auth resolvers shipped with the ai-sdk. Call this once at program startup before constructing a Runtime.

func RegisterClass

func RegisterClass(c ProviderClass)

RegisterClass registers a provider class. Calling RegisterClass with the same name twice is a no-op (the first registration wins).

func ResolveAPIKey

func ResolveAPIKey(cfg ProviderConfig) (string, error)

ResolveAPIKey extracts an API key from ProviderConfig, preferring the environment variable named by Auth.APIKeyEnv, then a literal APIKey.

Types

type AuthConfig

type AuthConfig struct {
	Type            AuthType `json:"type,omitempty"`
	APIKeyEnv       string   `json:"api_key_env,omitempty"`
	APIKey          string   `json:"api_key,omitempty"`
	AuthorizeURL    string   `json:"authorize_url,omitempty"`
	TokenURL        string   `json:"token_url,omitempty"`
	ClientID        string   `json:"client_id,omitempty"`
	IDP             string   `json:"idp,omitempty"`
	TokenAuthMethod string   `json:"token_auth_method,omitempty"`
}

AuthConfig describes how to obtain a bearer token or API key for a provider. Not all fields are used by every AuthType.

type AuthResolver

type AuthResolver interface {
	// Resolve returns a credential for the given provider configuration.
	Resolve(ctx context.Context, cfg ProviderConfig) (AuthResult, error)
}

AuthResolver resolves an AuthConfig into a credential at call time. Custom resolvers can be registered per class to handle bespoke auth flows (OAuth device code, short-lived token exchange, etc.).

func GetAuthResolver

func GetAuthResolver(t AuthType) (AuthResolver, bool)

GetAuthResolver returns the registered resolver for t, or false if none exists.

type AuthResolverFunc

type AuthResolverFunc func(ctx context.Context, cfg ProviderConfig) (AuthResult, error)

AuthResolverFunc adapts a function to the AuthResolver interface.

func (AuthResolverFunc) Resolve

Resolve implements AuthResolver.

type AuthResult

type AuthResult struct {
	Token   string
	Headers map[string]string
}

AuthResult is the resolved credential for a provider. It may be empty for providers that require no authentication.

func (AuthResult) IsEmpty

func (r AuthResult) IsEmpty() bool

IsEmpty reports whether the result carries no token or headers.

type AuthType

type AuthType string

AuthType names a provider authentication strategy.

const (
	AuthTypeNone      AuthType = "none"
	AuthTypeAPIKey    AuthType = "api_key"
	AuthTypeOAuthPKCE AuthType = "oauth_pkce"
)

Built-in auth types.

type Capability

type Capability string

Capability names a model capability a ProviderClass may satisfy.

const (
	CapabilityChat       Capability = "chat"
	CapabilityEmbed      Capability = "embed"
	CapabilityImage      Capability = "image"
	CapabilityVideo      Capability = "video"
	CapabilityObject     Capability = "object"
	CapabilityDecide     Capability = "decide"
	CapabilityRerank     Capability = "rerank"
	CapabilitySpeech     Capability = "speech"
	CapabilityTranscribe Capability = "transcribe"
	CapabilityMusic      Capability = "music"
)

Standard capabilities.

type Catalog

type Catalog = catalog.Catalog

The models.dev catalog lives in package catalog; these names keep the runtime's API.

func NewCatalog

func NewCatalog(opts CatalogOptions) *Catalog

NewCatalog returns an empty catalog configured by opts.

type CatalogModel

type CatalogModel = catalog.Model

The models.dev catalog lives in package catalog; these names keep the runtime's API.

type CatalogOptions

type CatalogOptions = catalog.Options

The models.dev catalog lives in package catalog; these names keep the runtime's API.

type CatalogProvider

type CatalogProvider = catalog.Provider

The models.dev catalog lives in package catalog; these names keep the runtime's API.

type Config

type Config struct {
	// DefaultProvider is the provider ID used when a model reference
	// does not include a provider prefix.
	DefaultProvider string `json:"default_provider,omitempty"`

	// Providers are the configured provider instances. The map key is
	// the local provider ID and must match ProviderConfig.ID.
	Providers map[string]ProviderConfig `json:"providers,omitempty"`

	// CatalogURL overrides the default models.dev endpoint. Empty means
	// use DefaultCatalogURL.
	CatalogURL string `json:"catalog_url,omitempty"`

	// CatalogCachePath enables on-disk caching of the fetched catalog.
	CatalogCachePath string `json:"catalog_cache_path,omitempty"`

	// HTTPTransport, when set, carries every provider request (for example
	// an otelhttp-wrapped transport). A ProviderConfig.Transport takes
	// precedence. Nil uses http.DefaultTransport.
	HTTPTransport http.RoundTripper `json:"-"`
}

Config is the declarative runtime configuration. Applications supply this (e.g. from JSON/YAML) and the Runtime turns it into working provider instances.

func (Config) ProviderByID

func (c Config) ProviderByID(id string) (ProviderConfig, bool)

ProviderByID returns the provider config with the given id, or false.

type CostConfig

type CostConfig struct {
	Input      float64 `json:"input,omitzero"`
	Output     float64 `json:"output,omitzero"`
	CacheRead  float64 `json:"cache_read,omitzero"`
	CacheWrite float64 `json:"cache_write,omitzero"`
}

CostConfig carries per-token pricing metadata.

type ModelConfig

type ModelConfig struct {
	ID               string `json:"id"`
	Name             string `json:"name,omitempty"`
	URL              string `json:"url,omitempty"`
	ContextWindow    int    `json:"context_window,omitzero"`
	MaxOutputTokens  int    `json:"max_output_tokens,omitzero"`
	Reasoning        bool   `json:"reasoning,omitzero"`
	ToolCall         bool   `json:"tool_call,omitzero"`
	StructuredOutput bool   `json:"structured_output,omitzero"`
	// Temperature reports whether the model accepts a temperature parameter.
	// nil means unspecified (unknown), true/false override catalog metadata
	// explicitly.
	Temperature *bool `json:"temperature,omitzero"`
	// Capabilities optionally declares the model's capabilities, overriding
	// metadata-derived ones. When non-empty it is authoritative: a capability
	// absent from the list is treated as unsupported even when catalog
	// modalities would imply it.
	Capabilities []Capability   `json:"capabilities,omitempty"`
	Cost         CostConfig     `json:"cost"`
	Extra        map[string]any `json:"extra,omitempty"`
}

ModelConfig is a configured model entry for a provider. It is merged with catalog metadata by the runtime.

type ModelInfo

type ModelInfo struct {
	ID               string
	ProviderID       string
	Name             string
	URL              string
	ContextWindow    int
	MaxOutputTokens  int
	Reasoning        bool
	ReasoningOptions []ReasoningOption
	ToolCall         bool
	StructuredOutput bool
	// Temperature reports whether the model accepts a temperature parameter.
	// nil means unknown (the catalog published no flag and no override set
	// one), false means the model rejects it.
	Temperature *bool
	// InputModalities and OutputModalities mirror the provider's published
	// modalities (for example models.dev's audio/speech, text, image, video).
	// Empty means unknown.
	InputModalities  []string
	OutputModalities []string
	// Capabilities, when non-empty, is the authoritative capability list and
	// overrides the modality-derived inference.
	Capabilities []Capability
	Cost         CostConfig
	Extra        map[string]any
}

ModelInfo is the runtime's normalised view of a model, combining catalog data and configured overrides.

type ModelRef

type ModelRef struct {
	ProviderID string
	ModelID    string
}

ModelRef is a parsed model reference of the form "provider/model" or just "model" when a default provider is configured.

type OAuthPKCEResolver

type OAuthPKCEResolver struct {
	// CacheDir overrides the directory used to store access tokens.
	// Defaults to os.UserConfigDir()/ai-sdk/oauth.
	CacheDir string

	// OpenBrowser wraps browser.OpenURL. Tests can override it.
	OpenBrowser func(url string) error
}

OAuthPKCEResolver is the built-in AuthResolver for AuthTypeOAuthPKCE. It is registered by RegisterBuiltinClasses so that providers configured with auth.type = "oauth_pkce" can obtain bearer tokens automatically.

func (*OAuthPKCEResolver) Resolve

Resolve implements AuthResolver.

type ProviderClass

type ProviderClass interface {
	// Name returns the stable class identifier used in ProviderConfig.Class.
	Name() string

	// Supports reports whether this class can satisfy cap.
	Supports(cap Capability) bool

	// New builds a ProviderSet from cfg for the given model. ctx may be
	// used for short-lived setup requests (e.g. discovery or token
	// exchange), but must not be stored.
	New(ctx context.Context, cfg ProviderConfig, model ModelInfo) (ProviderSet, error)
}

ProviderClass is a factory for provider instances. Each class knows how to turn a ProviderConfig (base URL, auth, options) and a resolved model into one or more domain providers.

func GetClass

func GetClass(name string) (ProviderClass, bool)

GetClass returns a registered class by name.

func MustRegisterClass

func MustRegisterClass(c ProviderClass) ProviderClass

MustRegisterClass is like RegisterClass but returns the class so it can be chained in var initializations.

type ProviderConfig

type ProviderConfig struct {
	// ID is the local identifier for this provider instance (e.g. "openai",
	// "my-maas"). It must be unique within a Runtime.
	ID string

	// Class selects the registered ProviderClass factory. Examples:
	// "openai-compatible", "anthropic-messages".
	Class string

	// BaseURL is the provider's API base URL. Individual models may
	// override this via their own URL field.
	BaseURL string

	// Auth describes how to obtain credentials for this provider.
	Auth AuthConfig

	// Headers are extra HTTP headers merged into every request.
	Headers map[string]string

	// Insecure disables TLS certificate verification.
	Insecure bool

	// Timeout is an optional whole-request timeout in milliseconds. Values
	// greater than zero opt in to a timeout; zero leaves requests unlimited so
	// callers can control cancellation through context deadlines.
	Timeout int

	// Transport carries this provider's HTTP requests. Nil falls back to
	// Config.HTTPTransport, then http.DefaultTransport.
	Transport http.RoundTripper `json:"-"`

	// Options carries class-specific options as a generic map. Classes
	// read only the keys they understand.
	Options map[string]any

	// Models allows per-provider model overrides/advertisements. These
	// augment or override the catalog when the runtime resolves model
	// references.
	Models []ModelConfig
}

ProviderConfig is the minimal information needed to construct a provider instance. The Class field selects which ProviderClass factory is invoked.

type ProviderSet

type ProviderSet struct {
	Chat       chat.Provider
	Embed      embed.Provider
	Image      image.Provider
	Video      video.Provider
	Object     object.Provider
	Decide     decide.Provider
	Rerank     rerank.Provider
	Speech     speech.Provider
	Transcribe transcribe.Provider
	Music      music.Provider
}

ProviderSet is a collection of domain providers produced by a single ProviderClass instance.

func (ProviderSet) Has

func (s ProviderSet) Has(cap Capability) bool

Has reports whether the set satisfies cap.

type ReasoningOption

type ReasoningOption = catalog.ReasoningOption

The models.dev catalog lives in package catalog; these names keep the runtime's API.

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime resolves model references to provider instances and exposes a high-level chat/embed API. It caches provider instances so repeated calls to the same provider/model/URL reuse the same underlying HTTP client.

func NewRuntime

func NewRuntime(cfg Config) *Runtime

NewRuntime creates a Runtime from the supplied configuration. It does not load the catalog automatically; call LoadCatalog (or supply a pre-populated Catalog) before making calls.

func NewRuntimeWithCatalog

func NewRuntimeWithCatalog(cfg Config, catalog *Catalog) *Runtime

NewRuntimeWithCatalog creates a Runtime that uses an already-populated catalog.

func (*Runtime) Catalog

func (r *Runtime) Catalog() *Catalog

Catalog returns the runtime's catalog. It may be nil until LoadCatalog is called.

func (*Runtime) Chat

Chat performs a non-streaming chat completion for the given model reference. The model field inside opts is overwritten with the resolved model ID.

func (*Runtime) ChatModelInfo added in v0.2.0

func (r *Runtime) ChatModelInfo(ref string) (chat.ModelInfo, error)

ChatModelInfo resolves a model reference to the request-shaping flags that belong on a chat request, without building the provider. Callers that resolve their own provider (for example agentloop) use it so their requests carry the same shaping flags as Chat and ChatStream.

func (*Runtime) ChatProvider

func (r *Runtime) ChatProvider(ctx context.Context, ref string) (chat.Provider, string, error)

ChatProvider resolves a model reference to a chat.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

func (*Runtime) ChatStream

func (r *Runtime) ChatStream(ctx context.Context, ref string, opts core.GenerateOptions) (core.StreamResult, error)

ChatStream performs a streaming chat completion for the given model reference.

func (*Runtime) Decide added in v0.2.0

func (r *Runtime) Decide(ctx context.Context, ref string, req decide.Request) (decide.Response, error)

Decide answers named questions using the resolved model.

func (*Runtime) DecideProvider added in v0.2.0

func (r *Runtime) DecideProvider(ctx context.Context, ref string) (decide.Provider, string, error)

DecideProvider resolves a decision provider, refusing published metadata that excludes decisions. Unknown compatible models are discovered from the server.

func (*Runtime) Image added in v0.1.31

Image performs a non-streaming image generation for the given model reference. The model field inside req is overwritten with the resolved model ID.

func (*Runtime) ImageProvider added in v0.1.31

func (r *Runtime) ImageProvider(ctx context.Context, ref string) (image.Provider, string, error)

ImageProvider resolves a model reference to an image.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

func (*Runtime) LoadCatalog

func (r *Runtime) LoadCatalog(ctx context.Context) error

LoadCatalog fetches or loads the models.dev catalog according to the runtime configuration. If cfg.CatalogURL or cfg.CatalogCachePath are set, they override the catalog defaults.

func (*Runtime) Models

func (r *Runtime) Models(providerID string) ([]ModelInfo, error)

Models returns the resolved model information for a provider, merged from configured overrides and the catalog.

func (*Runtime) Music added in v0.1.38

Music generates a music track for the given model reference. The model field inside req is overwritten with the resolved model ID.

func (*Runtime) MusicProvider added in v0.1.38

func (r *Runtime) MusicProvider(ctx context.Context, ref string) (music.Provider, string, error)

MusicProvider resolves a model reference to a music.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

func (*Runtime) Object added in v0.1.31

func (r *Runtime) Object(ctx context.Context, ref string, req object.Request) (object.ObjectResult, error)

Object performs a non-streaming object generation for the given model reference. The model field inside req is overwritten with the resolved model ID.

func (*Runtime) ObjectProvider added in v0.1.31

func (r *Runtime) ObjectProvider(ctx context.Context, ref string) (object.Provider, string, error)

ObjectProvider resolves a model reference to an object.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

func (*Runtime) ParseModelRef

func (r *Runtime) ParseModelRef(ref string) (ModelRef, error)

ParseModelRef splits a model reference into provider and model IDs. If the reference has no "/" separator and a default provider is configured, the default provider is used.

func (*Runtime) Rerank added in v0.1.31

func (r *Runtime) Rerank(ctx context.Context, ref string, req rerank.Request) (rerank.Response, error)

Rerank re-orders documents for the given model reference. The model field inside req is overwritten with the resolved model ID.

func (*Runtime) RerankProvider added in v0.1.31

func (r *Runtime) RerankProvider(ctx context.Context, ref string) (rerank.Provider, string, error)

RerankProvider resolves a model reference to a rerank.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

func (*Runtime) ResolveAuth

func (r *Runtime) ResolveAuth(ctx context.Context, providerID string) (AuthResult, error)

ResolveAuth resolves credentials for a configured provider using its registered auth resolver.

func (*Runtime) Speech added in v0.1.29

Speech performs a non-streaming text-to-speech generation for the given model reference. The model field inside req is overwritten with the resolved model ID.

func (*Runtime) SpeechProvider added in v0.1.29

func (r *Runtime) SpeechProvider(ctx context.Context, ref string) (speech.Provider, string, error)

SpeechProvider resolves a model reference to a speech.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

func (*Runtime) SpeechStream added in v0.1.39

SpeechStream starts a streaming text-to-speech generation for the given model reference. The model field inside req is overwritten with the resolved model ID, and the caller must Close the returned stream when finished.

func (*Runtime) SpeechStreamProvider added in v0.1.39

func (r *Runtime) SpeechStreamProvider(ctx context.Context, ref string) (speech.Streamer, string, error)

SpeechStreamProvider resolves a model reference to a speech.Streamer. It returns ErrCapabilityNotSupported when the resolved provider cannot stream.

func (*Runtime) Transcribe added in v0.1.29

Transcribe performs a non-streaming audio transcription for the given model reference. The model field inside req is overwritten with the resolved model ID.

func (*Runtime) TranscribeProvider added in v0.1.29

func (r *Runtime) TranscribeProvider(ctx context.Context, ref string) (transcribe.Provider, string, error)

TranscribeProvider resolves a model reference to a transcribe.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

func (*Runtime) Video added in v0.1.31

Video performs a non-streaming video generation for the given model reference. The model field inside req is overwritten with the resolved model ID.

func (*Runtime) VideoProvider added in v0.1.31

func (r *Runtime) VideoProvider(ctx context.Context, ref string) (video.Provider, string, error)

VideoProvider resolves a model reference to a video.Provider. It returns the provider instance and the resolved model ID that should be passed to requests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址