lkmedia

package
v0.16.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 5, 2026 License: MIT Imports: 29 Imported by: 0

Documentation

Overview

Package lkmedia implements the LiveKit signaling protocol and media transport for VoiceBlender's livekit_room leg type.

The signaling client (this file) speaks the LiveKit signaling protocol directly over a WebSocket using only the protobuf message types from github.com/livekit/protocol — no LiveKit SDK is imported. This keeps the pion stack pinned to VoiceBlender's own versions.

Index

Constants

View Source
const (
	DefaultSampleRate      = 48000 // gopus encoder/decoder fixed at 48 kHz
	DefaultFrameMs         = 20    // standard Opus ptime
	DefaultOpusBitrate     = 24000 // sensible voice bitrate; LiveKit defaults similar
	DefaultIngressBufferMs = 1000  // bound for the sum-mixed PCM ingress buffer
	DefaultLanePCMBufferMs = 200   // per-LK-remote-track decoded PCM ring (drop-oldest)
)

Defaults applied by Config.Validate when zero values are supplied.

Variables

This section is empty.

Functions

func MintJoinToken

func MintJoinToken(apiKey, apiSecret string, c JoinClaims) (string, error)

MintJoinToken returns a LiveKit-format JWT signed with apiSecret. The JWT shape matches the LiveKit Go server-sdk's output (HS256, `iss` = API key, `sub` = identity, `exp` from TTL, `video.*` grants).

Validation: apiKey, apiSecret, claims.Identity, and claims.Room must be non-empty. TTL <= 0 defaults to 6 hours (LiveKit's recommended max).

Types

type Callbacks

type Callbacks struct {
	// OnRemoteAudioTrack is the central callback for Model B. Fires once
	// per remote audio track the subscriber PC subscribes to. The API
	// layer is expected to spawn a LiveKitParticipantLeg using `pcm` as
	// the leg's AudioReader and add it to the umbrella's VB room. The
	// reader yields PCM16-LE at 48 kHz; reads return io.EOF when the
	// track ends or the transport closes.
	//
	// `identity` may be empty if pion fires OnTrack before the
	// corresponding ParticipantUpdate arrives. In that case the callback
	// also receives `participantSID` so callers can map it later.
	OnRemoteAudioTrack func(identity, participantSID, trackSID string, pcm io.Reader)

	// OnRemoteAudioTrackEnded fires when a previously-published track is
	// torn down (TrackUnpublished, ParticipantUpdate DISCONNECTED, or the
	// underlying RTP stream stops). The API layer cleans up the matching
	// participant leg.
	OnRemoteAudioTrackEnded func(trackSID string)

	OnParticipantsUpdated  func(participants []*livekit.ParticipantInfo)
	OnSpeakersChanged      func(speakers []*livekit.SpeakerInfo)
	OnTrackPublishedRemote func(participantSID string, track *livekit.TrackInfo)
	OnTrackUnpublishedSID  func(trackSID string)
	OnConnectionQuality    func(updates []*livekit.ConnectionQualityInfo)
	OnRefreshToken         func(token string)
}

Callbacks bundles the hooks the API layer wires up to bridge LK signaling state into VoiceBlender's leg/room model. Model B: each subscribed remote audio track becomes a LiveKitParticipantLeg via OnRemoteAudioTrack; the rest are observability events.

type Config

type Config struct {
	SampleRate      int
	FrameMs         int
	OpusBitrate     int
	IngressBufferMs int
	LaneBufferMs    int
	Log             *slog.Logger
}

Config configures a Transport. SampleRate is fixed at 48 kHz because gopus's encoder/decoder are constructed at 48 kHz mono; resampling to the room rate is the room's responsibility.

func (*Config) FrameBytesPCM

func (c *Config) FrameBytesPCM() int

FrameBytesPCM returns the number of bytes per frame in PCM16-LE.

func (*Config) FrameSamples

func (c *Config) FrameSamples() int

FrameSamples returns the number of PCM samples per frame.

func (*Config) IngressBufferBytes

func (c *Config) IngressBufferBytes() int

IngressBufferBytes is the byte capacity of the sum-mixed output buffer that AudioReader drains.

func (*Config) LaneCapacityFrames

func (c *Config) LaneCapacityFrames() int

LaneCapacityFrames is the per-remote-track ring capacity in frames.

func (*Config) Validate

func (c *Config) Validate() error

type JoinClaims

type JoinClaims struct {
	Identity       string
	Name           string
	Room           string
	TTL            time.Duration
	CanPublish     bool
	CanSubscribe   bool
	CanPublishData bool
	RoomAdmin      bool
	// Kind sets the participant kind reported by the LiveKit server
	// (ParticipantInfo.Kind). Empty defaults to STANDARD. Valid values
	// match the lower-case spelling LiveKit uses on the wire:
	// "agent", "ingress", "egress", "sip", "standard". Used to make
	// VoiceBlender show up as an AI-agent participant for clients like
	// livekit-examples/agent-starter-react that wait for a participant
	// of kind=agent before considering the session ready.
	Kind string
}

JoinClaims captures the LiveKit join-token fields VoiceBlender mints when the request omits a pre-signed Token and LIVEKIT_TOKEN_SIGNING is enabled. Mirrors the LiveKit JWT claim shape exactly so the resulting token is indistinguishable from one minted by the LiveKit CLI.

type PeerConfig

type PeerConfig struct {
	// ICEServers overrides the STUN/TURN list from JoinResponse. When non-empty,
	// VoiceBlender uses these instead of the server-supplied list.
	ICEServers []webrtc.ICEServer
	// RTPPortMin/RTPPortMax constrain pion's ephemeral UDP port range.
	// Both zero means use pion defaults.
	RTPPortMin, RTPPortMax int
}

PeerConfig configures the pion PeerConnections used by the transport.

type SignalClient

type SignalClient struct {
	// contains filtered or unexported fields
}

SignalClient is a connected LiveKit signaling session. Methods are safe for concurrent use; one client manages one WebSocket.

func Connect

func Connect(ctx context.Context, cfg SignalConfig) (*SignalClient, error)

Connect dials the LiveKit signaling WebSocket and completes the JOIN handshake. On success it returns a client that is already pumping SignalResponse messages into Events(). On failure it returns an error and no goroutines have been started.

func (*SignalClient) AddTrack

func (c *SignalClient) AddTrack(req *livekit.AddTrackRequest) error

AddTrack announces a new local track to be published. The server replies with a TrackPublished event whose Track.Sid is bound to req.Cid.

func (*SignalClient) Close

func (c *SignalClient) Close(reason livekit.DisconnectReason) error

Close sends a LeaveRequest and shuts down the WS. Idempotent.

func (*SignalClient) CloseReason

func (c *SignalClient) CloseReason() string

CloseReason returns a short string describing why the client closed — suitable for use as a leg.disconnected reason.

func (*SignalClient) Done

func (c *SignalClient) Done() <-chan struct{}

Done is closed when the client disconnects (server-side leave, network error, or local Close).

func (*SignalClient) Err

func (c *SignalClient) Err() error

Err returns the disconnect error, or nil if the disconnect was clean. Only meaningful after Done() is closed.

func (*SignalClient) Events

func (c *SignalClient) Events() <-chan SignalEvent

Events returns the receive-only channel of typed signal events. The channel is closed when the client disconnects.

func (*SignalClient) JoinResponse

func (c *SignalClient) JoinResponse() *livekit.JoinResponse

JoinResponse returns the server's JoinResponse from the handshake.

func (*SignalClient) MuteTrack

func (c *SignalClient) MuteTrack(sid string, muted bool) error

MuteTrack toggles the mute state of one of our published tracks.

func (*SignalClient) SendAnswer

func (c *SignalClient) SendAnswer(sdp webrtc.SessionDescription) error

SendAnswer sends the client's SDP answer.

func (*SignalClient) SendOffer

func (c *SignalClient) SendOffer(sdp webrtc.SessionDescription) error

SendOffer sends a client-initiated SDP offer (publisher peer connection).

func (*SignalClient) SendTrickle

func (c *SignalClient) SendTrickle(cand webrtc.ICECandidateInit, target livekit.SignalTarget) error

SendTrickle sends a local ICE candidate to the server.

type SignalConfig

type SignalConfig struct {
	// URL is the LiveKit server endpoint, e.g. "wss://lk.example.com" or
	// "ws://localhost:7880". The "/rtc" path is appended automatically;
	// any existing path is overwritten.
	URL string
	// Token is the LiveKit JWT carried on the access_token query string.
	Token string
	// AutoSubscribe, when true (the default), asks the server to subscribe
	// the participant to all published tracks. Set false only for advanced
	// flows that drive UpdateSubscription manually.
	AutoSubscribe *bool
	// Log receives debug/info diagnostics; defaults to slog.Default().
	Log *slog.Logger
}

SignalConfig configures the WebSocket dial.

type SignalEvent

type SignalEvent interface {
	// contains filtered or unexported methods
}

SignalEvent is the closed sum type of typed events emitted on SignalClient.Events.

type SignalEventAnswer

type SignalEventAnswer struct {
	SDP          webrtc.SessionDescription
	ID           uint32
	MidToTrackID map[string]string
}

SignalEventAnswer is the server's SDP answer to our publisher offer.

type SignalEventConnectionQuality

type SignalEventConnectionQuality struct {
	Updates []*livekit.ConnectionQualityInfo
}

SignalEventConnectionQuality is per-participant connection quality info.

type SignalEventLeave

type SignalEventLeave struct {
	Reason       livekit.DisconnectReason
	CanReconnect bool
}

SignalEventLeave is sent when the server tells us to leave.

type SignalEventMute

type SignalEventMute struct {
	SID   string
	Muted bool
}

SignalEventMute is a forced mute/unmute issued by the server for a local track.

type SignalEventOffer

type SignalEventOffer struct {
	SDP          webrtc.SessionDescription
	ID           uint32
	MidToTrackID map[string]string
}

SignalEventOffer is the server's SDP offer (server→client). Client must call SetRemoteDescription then send an Answer.

type SignalEventParticipantUpdate

type SignalEventParticipantUpdate struct {
	Participants []*livekit.ParticipantInfo
}

SignalEventParticipantUpdate is a participant joined/left/state-changed update.

type SignalEventRefreshToken

type SignalEventRefreshToken struct {
	Token string
}

SignalEventRefreshToken carries an updated JWT (server-issued) for long-lived sessions. We persist it but do not auto-reconnect in v1.

type SignalEventSpeakersChanged

type SignalEventSpeakersChanged struct {
	Speakers []*livekit.SpeakerInfo
}

SignalEventSpeakersChanged carries the active-speaker list.

type SignalEventTrackPublished

type SignalEventTrackPublished struct {
	CID   string
	Track *livekit.TrackInfo
}

SignalEventTrackPublished is the server's ack of an AddTrack we sent — the Track.Sid is now bound to our CID.

type SignalEventTrackUnpublished

type SignalEventTrackUnpublished struct {
	TrackSID string
}

SignalEventTrackUnpublished is the server notifying a track was removed.

type SignalEventTrickle

type SignalEventTrickle struct {
	Candidate webrtc.ICECandidateInit
	Target    livekit.SignalTarget
	Final     bool
}

SignalEventTrickle is a remote ICE candidate from the server.

type Transport

type Transport struct {
	// contains filtered or unexported fields
}

Transport wraps the pion publisher + subscriber PeerConnections plus the signaling client. It exposes PCM audio I/O for the VoiceBlender room mixer and surfaces LK-internal state via Callbacks.

func NewTransport

func NewTransport(ctx context.Context, cfg Config, signalCfg SignalConfig, peer PeerConfig, cb Callbacks) (*Transport, error)

NewTransport dials LiveKit, completes signaling JOIN, sets up both pion PeerConnections, and starts the audio pumps. On any error during setup the transport is closed and the error is returned without emitting events.

func (*Transport) AudioLevel

func (t *Transport) AudioLevel() (uint8, bool)

AudioLevel reports the level + voice-activity flag of the most recently encoded outgoing frame. Satisfies audioLevelProvider for the audioLevelInterceptor.

func (*Transport) AudioWriter

func (t *Transport) AudioWriter() io.Writer

AudioWriter accepts PCM16-LE bytes at 48 kHz. The publish leg's AudioWriter calls land here; sendLoop drains them, Opus-encodes, and publishes to the local LiveKit track. Model B: remote audio is no longer sum-mixed by the transport — each remote LK participant is its own VB leg via OnRemoteAudioTrack, so there is no transport-level AudioReader anymore.

func (*Transport) Close

func (t *Transport) Close(reason livekit.DisconnectReason) error

Close shuts down the transport (signaling Leave + PC tear-down). Idempotent.

func (*Transport) CloseClient

func (t *Transport) CloseClient() error

CloseClient is a convenience for callers that don't want to depend on the livekit protocol enums — equivalent to Close(CLIENT_INITIATED).

func (*Transport) CloseReason

func (t *Transport) CloseReason() string

CloseReason returns the short tag describing why the transport closed.

func (*Transport) Done

func (t *Transport) Done() <-chan struct{}

Done is closed when the transport disconnects.

func (*Transport) Err

func (t *Transport) Err() error

Err returns the disconnect error, or nil for clean shutdown.

func (*Transport) LocalIdentity

func (t *Transport) LocalIdentity() string

LocalIdentity returns the participant identity from the JoinResponse — useful for the leg's Headers() map.

func (*Transport) RoomName

func (t *Transport) RoomName() string

RoomName returns the LK room name from JoinResponse.

func (*Transport) SetCallbacks

func (t *Transport) SetCallbacks(cb Callbacks)

SetCallbacks replaces the observability callbacks. Used by callers that need the leg's ID inside the callbacks but only have it after NewTransport returns. Thread-safe.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址