wsmedia

package
v0.16.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 5, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package wsmedia provides a reusable WebSocket transport for bidirectional PCM audio plus text and control messages. It is consumed by the WebSocket leg type and is shaped so that vendor-specific Agent providers can later reuse the same plumbing.

Index

Constants

View Source
const (
	DefaultSampleRate      = 16000
	DefaultFrameMs         = 20
	DefaultWriteTimeout    = 5 * time.Second
	DefaultPingInterval    = 30 * time.Second
	DefaultIngressBufferMs = 1000
	DefaultTextBufferDepth = 50
)

Defaults applied by Config.Validate when zero values are supplied.

Variables

View Source
var ErrTextDisabled = errors.New("wsmedia: text channel disabled")

ErrTextDisabled is returned by WriteText when the transport was configured with TextEnabled=false.

Functions

This section is empty.

Types

type AudioCodec

type AudioCodec interface {
	// Encode converts a PCM16 frame to a WS payload + opcode.
	Encode(pcm []int16) (payload []byte, opCode ws.OpCode, err error)
	// Decode converts a received WS frame back to PCM16 samples. Returns
	// nil samples (no error) for frames the codec doesn't recognize as
	// audio (e.g. a text frame seen by a binary codec).
	Decode(opCode ws.OpCode, payload []byte) (pcm []int16, err error)
	// WireOpCode is the opcode the codec produces when encoding audio.
	WireOpCode() ws.OpCode
}

AudioCodec converts between mixer-facing PCM16 frames and the on-the-wire representation. Implementations are stateless and safe for concurrent use.

func CodecFromConfig

func CodecFromConfig(c Config) (AudioCodec, error)

CodecFromConfig returns the AudioCodec selected by the Config.

type Config

type Config struct {
	// SampleRate is the leg-native PCM rate. Must be one of 8000, 16000,
	// 24000, 48000.
	SampleRate int
	// SampleFormat selects the on-the-wire PCM encoding. Defaults to s16le.
	SampleFormat SampleFormat
	// WireFormat selects framing. Defaults to WireBinary.
	WireFormat WireFormat
	// FrameMs is the per-frame duration in milliseconds. Defaults to 20
	// (matches the mixer ptime).
	FrameMs int
	// ReadTimeout bounds inter-frame idle on the WS recv side. Defaults to
	// wsutilx.DefaultReadTimeout (~60s).
	ReadTimeout time.Duration
	// WriteTimeout bounds how long a single WS write may block on TCP
	// backpressure before being failed. Defaults to 5s.
	WriteTimeout time.Duration
	// PingInterval is how often the server-side helper sends control pings.
	// Defaults to 30s.
	PingInterval time.Duration
	// IngressBufferMs caps how much inbound audio may be buffered before
	// the recv loop starts dropping frames. Defaults to 1000ms.
	IngressBufferMs int
	// JitterBufferMs is the ingress playout lead: AudioReader withholds audio
	// until this much is buffered, so a producer that runs late against the
	// mixer tick spends the lead instead of leaving a gap. 0 (default) is
	// passthrough. IngressBufferMs remains the capacity bound.
	JitterBufferMs int
	// TextBufferDepth caps how many inbound text messages may be buffered
	// before the recv loop starts dropping. Defaults to 50.
	TextBufferDepth int
	// Headers carries HTTP headers either to send (DialClient) or already
	// observed (UpgradeServer surfaces these via its return value).
	Headers http.Header
	// TextEnabled controls whether inbound and outbound text frames are
	// processed. Defaults to true.
	TextEnabled *bool
	// Log receives structured diagnostics. Required.
	Log *slog.Logger
}

Config configures a Transport. Both DialClient and UpgradeServer take this struct; required vs optional is documented per field.

func (*Config) FrameBytesPCM

func (c *Config) FrameBytesPCM() int

FrameBytesPCM returns the number of bytes per frame in the internal PCM16 representation (mixer-facing) regardless of wire SampleFormat.

func (*Config) FrameSamples

func (c *Config) FrameSamples() int

FrameSamples returns the number of PCM samples per frame.

func (*Config) IngressBufferBytes

func (c *Config) IngressBufferBytes() int

IngressBufferBytes is the byte capacity of the audio ingress streamBuffer.

func (*Config) JitterPlayoutBytes added in v0.13.0

func (c *Config) JitterPlayoutBytes() int

JitterPlayoutBytes is the ingress playout lead in whole frames, 0 when jitter buffering is disabled. A lead shorter than one frame is rounded up: asking for a buffer and getting none would be worse than the rounding.

func (*Config) TextEnabledValue

func (c *Config) TextEnabledValue() bool

TextEnabledValue dereferences TextEnabled with a true default. Use after Validate.

func (*Config) Validate

func (c *Config) Validate() error

Validate applies defaults and reports any unrecoverable misconfiguration. Callers should invoke this before DialClient/UpgradeServer.

type DriftStats added in v0.13.0

type DriftStats struct {
	// Underruns is how many times the lead ran dry and had to be rebuilt.
	Underruns int64
	// Trimmed and Inserted count frames removed or duplicated during a pause,
	// which is the free way to absorb drift: nothing audible changes.
	Trimmed  int64
	Inserted int64
	// Concealed counts invented frames emitted mid-speech, when the deficit
	// could not wait for a pause. These are audible in principle, unlike
	// Trimmed/Inserted.
	Concealed int64
	// Rewarms counts talkspurts that had the whole lead rebuilt first.
	Rewarms int64
}

DriftStats counts the corrections the playout buffer made to keep the producer's clock aligned with the mixer's.

type SampleFormat

type SampleFormat string

SampleFormat selects the on-the-wire PCM sample encoding. v1 only ships signed 16-bit little-endian; the type is enum-shaped so 8-bit / companded formats can be added later without API churn.

const (
	SampleS16LE SampleFormat = "s16le"
)

type Transport

type Transport struct {
	// contains filtered or unexported fields
}

Transport carries bidirectional PCM audio, text messages, and structured control frames over a single WebSocket connection. Constructed via DialClient (outbound) or UpgradeServer (inbound).

func DialClient

func DialClient(ctx context.Context, url string, cfg Config) (*Transport, http.Header, error)

DialClient opens a WebSocket connection to url and returns a Transport configured for client-side framing. The supplied cfg.Headers are sent on the upgrade request. The returned http.Header is the server's response header.

func UpgradeServer

func UpgradeServer(w http.ResponseWriter, r *http.Request, cfg Config) (*Transport, http.Header, error)

UpgradeServer turns an incoming HTTP request into a Transport on the server side of a WebSocket conversation. The returned http.Header is a copy of the request headers as observed before upgrade (callers commonly filter this to surface X-/P- custom headers).

On upgrade failure UpgradeServer has already written an HTTP error response to w.

func (*Transport) AudioDrift added in v0.13.0

func (t *Transport) AudioDrift() DriftStats

AudioDrift reports the corrections the ingress playout buffer made to keep the producer's clock aligned with the mixer's. All zero unless Config.JitterBufferMs is set.

func (*Transport) AudioDropsBytes

func (t *Transport) AudioDropsBytes() int64

AudioDropsBytes reports the cumulative count of inbound audio bytes discarded on ingress buffer overflow.

func (*Transport) AudioReader

func (t *Transport) AudioReader() io.Reader

AudioReader returns an io.Reader that yields inbound PCM16-LE mono at cfg.SampleRate, paced one frame per cfg.FrameMs. Suitable as a leg's AudioReader().

func (*Transport) Close

func (t *Transport) Close() error

Close shuts the transport down: cancels the context, closes the inbound audio buffer, closes the underlying conn, and — if audioOut was set via Start and implements io.Closer — closes that too so the send loop unblocks from any pending read. Safe to call repeatedly and from any goroutine.

func (*Transport) Done

func (t *Transport) Done() <-chan struct{}

Done returns a channel closed once Start's loops have all exited.

func (*Transport) Err

func (t *Transport) Err() error

Err returns the terminal error, if any. Stable after Done is closed.

func (*Transport) PeerHeaders

func (t *Transport) PeerHeaders() http.Header

PeerHeaders returns the HTTP headers exchanged at the handshake — for outbound dials, the server's response headers; for inbound upgrades, the client's request headers.

func (*Transport) SendStructured

func (t *Transport) SendStructured(v any) error

SendStructured marshals v to JSON and ships it as a WS text frame. Intended for callers (future agent providers) that need to send vendor-specific control messages outside the built-in envelope.

func (*Transport) SetOnControl

func (t *Transport) SetOnControl(fn func(raw json.RawMessage))

SetOnControl registers the callback invoked for inbound JSON text frames whose "type" isn't recognized by Transport itself. Lets agent providers plug in vendor-specific decoders.

func (*Transport) SetOnText

func (t *Transport) SetOnText(fn func(text string))

SetOnText registers the callback invoked for inbound text messages. Setting nil disables delivery (messages still count toward TextDrops).

func (*Transport) Start

func (t *Transport) Start(audioOut io.Reader)

Start begins the send/recv/ping goroutines. audioOut supplies PCM16-LE mono at cfg.SampleRate that will be encoded and shipped to the peer. Calling Start more than once is a no-op after the first call.

func (*Transport) TextDropsCount

func (t *Transport) TextDropsCount() int64

TextDropsCount reports the cumulative count of inbound text messages discarded because the text-channel callback wasn't draining fast enough.

func (*Transport) WriteText

func (t *Transport) WriteText(_ context.Context, text string) error

WriteText sends a text message to the peer as a JSON control frame. Returns ErrTextDisabled if the transport was constructed with text off.

type WireFormat

type WireFormat string

WireFormat selects how audio frames are framed on the WebSocket.

const (
	// WireBinary sends PCM as raw WebSocket binary frames. Most efficient;
	// used by Deepgram, VAPI, Pipecat-style providers.
	WireBinary WireFormat = "binary"
	// WireJSONBase64 wraps PCM in {"type":"audio","audio":"<b64>"} text
	// frames. Matches the existing /v1/rooms/{id}/ws shape; browser-friendly.
	WireJSONBase64 WireFormat = "json_base64"
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址