Documentation
¶
Overview ¶
Package wsmedia provides a reusable WebSocket transport for bidirectional PCM audio plus text and control messages. It is consumed by the WebSocket leg type and is shaped so that vendor-specific Agent providers can later reuse the same plumbing.
Index ¶
- Constants
- Variables
- type AudioCodec
- type Config
- type DriftStats
- type SampleFormat
- type Transport
- func (t *Transport) AudioDrift() DriftStats
- func (t *Transport) AudioDropsBytes() int64
- func (t *Transport) AudioReader() io.Reader
- func (t *Transport) Close() error
- func (t *Transport) Done() <-chan struct{}
- func (t *Transport) Err() error
- func (t *Transport) PeerHeaders() http.Header
- func (t *Transport) SendStructured(v any) error
- func (t *Transport) SetOnControl(fn func(raw json.RawMessage))
- func (t *Transport) SetOnText(fn func(text string))
- func (t *Transport) Start(audioOut io.Reader)
- func (t *Transport) TextDropsCount() int64
- func (t *Transport) WriteText(_ context.Context, text string) error
- type WireFormat
Constants ¶
const ( DefaultSampleRate = 16000 DefaultFrameMs = 20 DefaultWriteTimeout = 5 * time.Second DefaultPingInterval = 30 * time.Second DefaultIngressBufferMs = 1000 DefaultTextBufferDepth = 50 )
Defaults applied by Config.Validate when zero values are supplied.
Variables ¶
var ErrTextDisabled = errors.New("wsmedia: text channel disabled")
ErrTextDisabled is returned by WriteText when the transport was configured with TextEnabled=false.
Functions ¶
This section is empty.
Types ¶
type AudioCodec ¶
type AudioCodec interface {
// Encode converts a PCM16 frame to a WS payload + opcode.
Encode(pcm []int16) (payload []byte, opCode ws.OpCode, err error)
// Decode converts a received WS frame back to PCM16 samples. Returns
// nil samples (no error) for frames the codec doesn't recognize as
// audio (e.g. a text frame seen by a binary codec).
Decode(opCode ws.OpCode, payload []byte) (pcm []int16, err error)
// WireOpCode is the opcode the codec produces when encoding audio.
WireOpCode() ws.OpCode
}
AudioCodec converts between mixer-facing PCM16 frames and the on-the-wire representation. Implementations are stateless and safe for concurrent use.
func CodecFromConfig ¶
func CodecFromConfig(c Config) (AudioCodec, error)
CodecFromConfig returns the AudioCodec selected by the Config.
type Config ¶
type Config struct {
// SampleRate is the leg-native PCM rate. Must be one of 8000, 16000,
// 24000, 48000.
SampleRate int
// SampleFormat selects the on-the-wire PCM encoding. Defaults to s16le.
SampleFormat SampleFormat
// WireFormat selects framing. Defaults to WireBinary.
WireFormat WireFormat
// FrameMs is the per-frame duration in milliseconds. Defaults to 20
// (matches the mixer ptime).
FrameMs int
// ReadTimeout bounds inter-frame idle on the WS recv side. Defaults to
// wsutilx.DefaultReadTimeout (~60s).
ReadTimeout time.Duration
// WriteTimeout bounds how long a single WS write may block on TCP
// backpressure before being failed. Defaults to 5s.
WriteTimeout time.Duration
// PingInterval is how often the server-side helper sends control pings.
// Defaults to 30s.
PingInterval time.Duration
// IngressBufferMs caps how much inbound audio may be buffered before
// the recv loop starts dropping frames. Defaults to 1000ms.
IngressBufferMs int
// JitterBufferMs is the ingress playout lead: AudioReader withholds audio
// until this much is buffered, so a producer that runs late against the
// mixer tick spends the lead instead of leaving a gap. 0 (default) is
// passthrough. IngressBufferMs remains the capacity bound.
JitterBufferMs int
// TextBufferDepth caps how many inbound text messages may be buffered
// before the recv loop starts dropping. Defaults to 50.
TextBufferDepth int
// Headers carries HTTP headers either to send (DialClient) or already
// observed (UpgradeServer surfaces these via its return value).
Headers http.Header
// TextEnabled controls whether inbound and outbound text frames are
// processed. Defaults to true.
TextEnabled *bool
// Log receives structured diagnostics. Required.
Log *slog.Logger
}
Config configures a Transport. Both DialClient and UpgradeServer take this struct; required vs optional is documented per field.
func (*Config) FrameBytesPCM ¶
FrameBytesPCM returns the number of bytes per frame in the internal PCM16 representation (mixer-facing) regardless of wire SampleFormat.
func (*Config) FrameSamples ¶
FrameSamples returns the number of PCM samples per frame.
func (*Config) IngressBufferBytes ¶
IngressBufferBytes is the byte capacity of the audio ingress streamBuffer.
func (*Config) JitterPlayoutBytes ¶ added in v0.13.0
JitterPlayoutBytes is the ingress playout lead in whole frames, 0 when jitter buffering is disabled. A lead shorter than one frame is rounded up: asking for a buffer and getting none would be worse than the rounding.
func (*Config) TextEnabledValue ¶
TextEnabledValue dereferences TextEnabled with a true default. Use after Validate.
type DriftStats ¶ added in v0.13.0
type DriftStats struct {
// Underruns is how many times the lead ran dry and had to be rebuilt.
Underruns int64
// Trimmed and Inserted count frames removed or duplicated during a pause,
// which is the free way to absorb drift: nothing audible changes.
Trimmed int64
Inserted int64
// Concealed counts invented frames emitted mid-speech, when the deficit
// could not wait for a pause. These are audible in principle, unlike
// Trimmed/Inserted.
Concealed int64
// Rewarms counts talkspurts that had the whole lead rebuilt first.
Rewarms int64
}
DriftStats counts the corrections the playout buffer made to keep the producer's clock aligned with the mixer's.
type SampleFormat ¶
type SampleFormat string
SampleFormat selects the on-the-wire PCM sample encoding. v1 only ships signed 16-bit little-endian; the type is enum-shaped so 8-bit / companded formats can be added later without API churn.
const (
SampleS16LE SampleFormat = "s16le"
)
type Transport ¶
type Transport struct {
// contains filtered or unexported fields
}
Transport carries bidirectional PCM audio, text messages, and structured control frames over a single WebSocket connection. Constructed via DialClient (outbound) or UpgradeServer (inbound).
func DialClient ¶
DialClient opens a WebSocket connection to url and returns a Transport configured for client-side framing. The supplied cfg.Headers are sent on the upgrade request. The returned http.Header is the server's response header.
func UpgradeServer ¶
func UpgradeServer(w http.ResponseWriter, r *http.Request, cfg Config) (*Transport, http.Header, error)
UpgradeServer turns an incoming HTTP request into a Transport on the server side of a WebSocket conversation. The returned http.Header is a copy of the request headers as observed before upgrade (callers commonly filter this to surface X-/P- custom headers).
On upgrade failure UpgradeServer has already written an HTTP error response to w.
func (*Transport) AudioDrift ¶ added in v0.13.0
func (t *Transport) AudioDrift() DriftStats
AudioDrift reports the corrections the ingress playout buffer made to keep the producer's clock aligned with the mixer's. All zero unless Config.JitterBufferMs is set.
func (*Transport) AudioDropsBytes ¶
AudioDropsBytes reports the cumulative count of inbound audio bytes discarded on ingress buffer overflow.
func (*Transport) AudioReader ¶
AudioReader returns an io.Reader that yields inbound PCM16-LE mono at cfg.SampleRate, paced one frame per cfg.FrameMs. Suitable as a leg's AudioReader().
func (*Transport) Close ¶
Close shuts the transport down: cancels the context, closes the inbound audio buffer, closes the underlying conn, and — if audioOut was set via Start and implements io.Closer — closes that too so the send loop unblocks from any pending read. Safe to call repeatedly and from any goroutine.
func (*Transport) Done ¶
func (t *Transport) Done() <-chan struct{}
Done returns a channel closed once Start's loops have all exited.
func (*Transport) PeerHeaders ¶
PeerHeaders returns the HTTP headers exchanged at the handshake — for outbound dials, the server's response headers; for inbound upgrades, the client's request headers.
func (*Transport) SendStructured ¶
SendStructured marshals v to JSON and ships it as a WS text frame. Intended for callers (future agent providers) that need to send vendor-specific control messages outside the built-in envelope.
func (*Transport) SetOnControl ¶
func (t *Transport) SetOnControl(fn func(raw json.RawMessage))
SetOnControl registers the callback invoked for inbound JSON text frames whose "type" isn't recognized by Transport itself. Lets agent providers plug in vendor-specific decoders.
func (*Transport) SetOnText ¶
SetOnText registers the callback invoked for inbound text messages. Setting nil disables delivery (messages still count toward TextDrops).
func (*Transport) Start ¶
Start begins the send/recv/ping goroutines. audioOut supplies PCM16-LE mono at cfg.SampleRate that will be encoded and shipped to the peer. Calling Start more than once is a no-op after the first call.
func (*Transport) TextDropsCount ¶
TextDropsCount reports the cumulative count of inbound text messages discarded because the text-channel callback wasn't draining fast enough.
type WireFormat ¶
type WireFormat string
WireFormat selects how audio frames are framed on the WebSocket.
const ( // WireBinary sends PCM as raw WebSocket binary frames. Most efficient; // used by Deepgram, VAPI, Pipecat-style providers. WireBinary WireFormat = "binary" // WireJSONBase64 wraps PCM in {"type":"audio","audio":"<b64>"} text // frames. Matches the existing /v1/rooms/{id}/ws shape; browser-friendly. WireJSONBase64 WireFormat = "json_base64" )