Documentation
¶
Overview ¶
Package rtc connects the audio package to WebRTC tracks.
TrackWriter takes PCM and publishes it: it implements track.AudioSampleProvider, so it plugs straight into track.NewAudioTrack. TrackReader does the reverse, turning a remote track's RTP into PCM.
Between them a voice agent needs no codec or RTP code of its own: write what the text-to-speech engine produced, read what the participant said.
Index ¶
- Constants
- func NewAudioTrack(trackInfo *sfu_models.TrackInfo, w *TrackWriter, opts ...track.Option) (*track.Local, error)
- type RTPSource
- type ReaderConfig
- type TrackReader
- type TrackWriter
- func (w *TrackWriter) Buffered() time.Duration
- func (w *TrackWriter) Clear()
- func (w *TrackWriter) Close() error
- func (w *TrackWriter) Codec() webrtc.RTPCodecCapability
- func (w *TrackWriter) CurrentAudioLevel() uint8
- func (w *TrackWriter) Flush() error
- func (w *TrackWriter) NextSample(ctx context.Context) (media.Sample, error)
- func (w *TrackWriter) Write(pcm audio.PCM) error
- type WriterConfig
Constants ¶
const DefaultBufferDuration = 30 * time.Second
DefaultBufferDuration is how much encoded audio a TrackWriter holds before it starts dropping the oldest. It is generous because a text-to-speech engine commonly produces a whole sentence faster than real time.
const DefaultMaxConceal = 500 * time.Millisecond
DefaultMaxConceal caps how long a run of lost packets is concealed before the reader gives up and resynchronises. Beyond a few hundred milliseconds concealment stops sounding like the speaker and starts sounding like a fault, and the sender has probably stopped rather than been dropped.
const OpusClockRate = 48000
OpusClockRate is the RTP clock rate for Opus. RFC 7587 fixes it at 48 kHz regardless of the rate the codec is actually running at internally.
Variables ¶
This section is empty.
Functions ¶
func NewAudioTrack ¶
func NewAudioTrack(trackInfo *sfu_models.TrackInfo, w *TrackWriter, opts ...track.Option) (*track.Local, error)
NewAudioTrack builds a local audio track fed by w, wiring up the Opus codec parameters so the caller does not have to restate them.
Types ¶
type RTPSource ¶
type RTPSource interface {
ReadRTP() (*rtp.Packet, interceptor.Attributes, error)
}
RTPSource is the part of webrtc.TrackRemote a TrackReader uses. Taking an interface lets a reader run over a recorded stream or a test fixture as well as a live track.
type ReaderConfig ¶
type ReaderConfig struct {
// Opus configures the decoder. Its zero value is 48 kHz mono at 20 ms.
// Set SampleRate to what your consumer wants, typically 16000 for speech
// recognition, and the decoder produces it directly instead of making you
// resample afterwards.
Opus opus.Config
// DisableConcealment stops the reader from synthesising audio for lost
// packets. By default a gap in sequence numbers is filled in, which keeps
// the output continuous and lets the decoder fade rather than click.
DisableConcealment bool
// MaxConceal caps a run of concealed audio. Defaults to
// [DefaultMaxConceal].
MaxConceal time.Duration
}
ReaderConfig configures a TrackReader. The zero value decodes to 48 kHz mono with concealment enabled.
type TrackReader ¶
type TrackReader struct {
// contains filtered or unexported fields
}
TrackReader turns a remote WebRTC audio track into PCM.
It reads RTP, unwraps redundancy when the track carries it, decodes Opus, and fills gaps left by lost packets. TrackReader.Read returns one buffer per call; TrackReader.Frames wraps it as an iterator for a range loop.
A TrackReader is not safe for concurrent use. Run one per track, which is the natural shape anyway since each track arrives on its own callback.
func NewRTPReader ¶
func NewRTPReader(src RTPSource, codec webrtc.RTPCodecCapability, cfg ReaderConfig) (*TrackReader, error)
NewRTPReader returns a reader over any source of Opus RTP packets. codec says what the packets carry, which decides whether they need unwrapping from RED and what clock rate their timestamps use.
func NewTrackReader ¶
func NewTrackReader(remote *webrtc.TrackRemote, cfg ReaderConfig) (*TrackReader, error)
NewTrackReader returns a reader for a remote audio track.
The track comes from the OnTrack callback the SDK invokes when a subscription is established; pass its Track field.
func (*TrackReader) Close ¶
func (r *TrackReader) Close() error
Close releases the reader. The remote track itself is owned by the SDK and is not closed here.
type TrackWriter ¶
type TrackWriter struct {
track.BaseSampleProvider
// contains filtered or unexported fields
}
TrackWriter publishes PCM to a WebRTC audio track.
Call TrackWriter.Write with audio at any sample rate, channel count and format; it is resampled, encoded and queued. The track pulls packets off the queue in real time, so Write never blocks on the network.
When the queue runs dry the writer emits encoded silence rather than stalling, which keeps the RTP timeline continuous. That matters: a receiver that sees a gap in timestamps treats it as loss and starts concealing.
A TrackWriter is safe for concurrent use.
func NewTrackWriter ¶
func NewTrackWriter(cfg WriterConfig) (*TrackWriter, error)
NewTrackWriter returns a writer for cfg.
func (*TrackWriter) Buffered ¶
func (w *TrackWriter) Buffered() time.Duration
Buffered is how much encoded audio is waiting to be sent.
func (*TrackWriter) Clear ¶
func (w *TrackWriter) Clear()
Clear drops everything queued and resets the encoder.
This is barge-in: when a participant interrupts the agent, whatever it was about to say has to stop immediately, and a buffer holding several seconds of speech would otherwise keep playing.
func (*TrackWriter) Close ¶
func (w *TrackWriter) Close() error
Close stops the track's write loop. Queued audio is discarded.
func (*TrackWriter) Codec ¶
func (w *TrackWriter) Codec() webrtc.RTPCodecCapability
Codec is the RTP codec capability this writer produces, ready to hand to track.NewAudioTrack. RFC 7587 signals Opus as opus/48000/2 whatever the encoder's channel count, and a mono stream is a valid stream of that codec, so the capability always says two channels: anything else fails to bind to the codec every peer negotiates.
func (*TrackWriter) CurrentAudioLevel ¶
func (w *TrackWriter) CurrentAudioLevel() uint8
CurrentAudioLevel implements track.AudioSampleProvider. It reports the loudness of the packet most recently handed to the track, on the WebRTC scale where 0 is full scale and 127 is silence.
func (*TrackWriter) Flush ¶
func (w *TrackWriter) Flush() error
Flush pushes out the partial frame the encoder is holding, padded with silence. Call it when a source has finished, so its last few milliseconds are not left waiting for audio that will never come.
func (*TrackWriter) NextSample ¶
NextSample implements track.SampleProvider. The track calls it on its own goroutine and paces the calls off the returned duration, so it returns immediately with silence rather than waiting for audio to arrive.
func (*TrackWriter) Write ¶
func (w *TrackWriter) Write(pcm audio.PCM) error
Write encodes pcm and queues it for sending. It returns as soon as the audio is queued, not when it has been sent.
Audio that does not fill a whole frame stays in the encoder until the next call. Use TrackWriter.Flush at the end of an utterance to push it out.
type WriterConfig ¶
type WriterConfig struct {
// Opus configures the encoder. Its zero value is 48 kHz mono at 20 ms,
// sent as the opus/48000/2 codec that WebRTC negotiates.
Opus opus.Config
// BufferDuration caps how much encoded audio is queued. Once it is
// exceeded the oldest audio is dropped, which bounds both memory and the
// delay between writing a sample and hearing it. Defaults to
// [DefaultBufferDuration].
BufferDuration time.Duration
}
WriterConfig configures a TrackWriter. The zero value is a working 48 kHz mono voice configuration.
Source Files