Documentation
¶
Overview ¶
Package output implements Sypl outputs: an output processes, and writes a message to its writer - anything implementing io.Writer.
Built-ins: Console (stdout), StdErr (error levels to stderr), File, and FileBased (files), SafeBuffer (an in-memory concurrent-safe buffer), ElasticSearch (one request per document), and the reliability set:
- Async: wraps ANY output into a bounded, buffered, asynchronous one - a single worker drains the buffer preserving FIFO order, with Block, DropNewest, or DropOldest full-buffer policies.
- ElasticSearchBulk (and ...WithDynamicIndex): batches documents into _bulk requests via esutil's BulkIndexer - the high-throughput sibling of ElasticSearch.
- RotatingFile: a file output with native size-based rotation, backup timestamping, and count/age pruning.
- Recorder: captures structured snapshots of everything written - a test-assertion helper for Sypl consumers.
Outputs that buffer implement `Flush() error`; outputs owning resources implement `Close() error` (io.Closer). Close is idempotent, and writes after Close return a typed, per-output sentinel error - never a panic. Flush after Close is a no-op.
Index ¶
- Variables
- func Recorder(maxLevel level.Level, processors ...processor.IProcessor) (*RecorderOutput, IOutput)
- type AsyncOption
- type AsyncPolicy
- type ElasticSearchBulkOption
- type ElasticSearchConfig
- type ElasticSearchDynamicIndexFunc
- type ElasticSearchTagMap
- type ElasticSearchTagMapItem
- type IOutput
- func Async(o IOutput, opts ...AsyncOption) IOutput
- func Console(maxLevel level.Level, processors ...processor.IProcessor) IOutput
- func ElasticSearch(indexName string, esConfig ElasticSearchConfig, maxLevel level.Level, ...) IOutput
- func ElasticSearchBulk(indexName string, esConfig ElasticSearchConfig, maxLevel level.Level, ...) IOutput
- func ElasticSearchBulkWithDynamicIndex(dynamicIndexFunc ElasticSearchDynamicIndexFunc, esConfig ElasticSearchConfig, ...) IOutput
- func ElasticSearchWithDynamicIndex(dynamicIndexFunc ElasticSearchDynamicIndexFunc, esConfig ElasticSearchConfig, ...) IOutput
- func ElasticSearchWithTagMap(tagMap ElasticSearchTagMap, esConfig ElasticSearchConfig, ...) []IOutput
- func File(name string, path string, maxLevel level.Level, ...) IOutput
- func FileBased(name string, maxLevel level.Level, writer io.Writer, ...) IOutput
- func New(name string, maxLevel level.Level, w io.Writer, ...) IOutput
- func RotatingFile(name, path string, maxLevel level.Level, cfg RotationConfig, ...) (IOutput, error)
- func SafeBuffer(maxLevel level.Level, processors ...processor.IProcessor) (*safebuffer.Buffer, IOutput)
- func StdErr(processors ...processor.IProcessor) IOutput
- type Record
- type RecorderOutput
- func (p RecorderOutput) AddProcessors(processors ...processor.IProcessor) IOutput
- func (p RecorderOutput) GetBuiltinLogger() *builtin.Builtin
- func (p RecorderOutput) GetFormatter() formatter.IFormatter
- func (p RecorderOutput) GetMaxLevel() level.Level
- func (p RecorderOutput) GetName() string
- func (p RecorderOutput) GetProcessor(name string) processor.IProcessor
- func (p RecorderOutput) GetProcessors() []processor.IProcessor
- func (p RecorderOutput) GetProcessorsNames() []string
- func (p RecorderOutput) GetStatus() status.Status
- func (p RecorderOutput) GetWriter() io.Writer
- func (r *RecorderOutput) Len() int
- func (r *RecorderOutput) Messages() []Record
- func (r *RecorderOutput) Reset()
- func (p RecorderOutput) SetBuiltinLogger(builtinLogger *builtin.Builtin) IOutput
- func (p RecorderOutput) SetFormatter(fmtr formatter.IFormatter) IOutput
- func (p RecorderOutput) SetMaxLevel(l level.Level) IOutput
- func (p RecorderOutput) SetName(name string)
- func (p RecorderOutput) SetProcessors(processors ...processor.IProcessor) IOutput
- func (p RecorderOutput) SetStatus(s status.Status)
- func (p RecorderOutput) SetWriter(w io.Writer) IOutput
- func (p RecorderOutput) String() string
- func (r *RecorderOutput) Write(m message.IMessage) error
- type RotationConfig
Constants ¶
This section is empty.
Variables ¶
var ( // ErrAsyncClosed is returned when writing to a closed async output. ErrAsyncClosed = errors.New("async output is closed") // ErrAsyncDropped is the sentinel wrapped into drop notifications // delivered to the error handler. Check it with `errors.Is`. ErrAsyncDropped = errors.New("async output dropped a message") )
var ErrRotatingFileClosed = errors.New("rotating file output is closed")
ErrRotatingFileClosed is returned when writing to a closed rotating file output.
ErrRotatingFileUnavailable is returned - wrapped with the underlying cause - while the rotating writer has no open live file: a mid-rotation failure closed it, and reopening the original path failed too. Every write (and Flush) retries the reopen first, so the output self-heals once the filesystem recovers. Check it with `errors.Is`.
NOTE: Deliberately NOT `os.ErrClosed`-classed: the output layer treats `os.ErrClosed` as a benign closed-writer condition, and swallows it - this error must surface (e.g. through `Sypl.SetErrorHandler`).
Functions ¶
func Recorder ¶ added in v1.21.0
func Recorder(maxLevel level.Level, processors ...processor.IProcessor) (*RecorderOutput, IOutput)
Recorder is a built-in `output` - named `Recorder` - capturing a structured snapshot (level, original, and processed content, fields, tags, output, and processors names, timestamp) of every message it actually writes: level gating, flags, and processors are applied through the standard pipeline first. Designed for test assertions by Sypl consumers.
It returns both the concrete recorder - for `Messages`, `Len`, and `Reset` - and the `IOutput` to register on the logger; they're the same instance. Thread-safe.
Types ¶
type AsyncOption ¶ added in v1.21.0
type AsyncOption func(*asyncOutput)
AsyncOption configures the async output wrapper.
func AsyncWithBufferSize ¶ added in v1.21.0
func AsyncWithBufferSize(size int) AsyncOption
AsyncWithBufferSize sets the buffer capacity. Non-positive values fall back to the default (1024).
func AsyncWithErrorHandler ¶ added in v1.21.0
func AsyncWithErrorHandler(handler func(error)) AsyncOption
AsyncWithErrorHandler sets the handler receiving the wrapped output's write errors, drop notifications (wrapping `ErrAsyncDropped`), interval-flush errors, and panics from the wrapped output - converted to errors, so a misbehaving sink never kills the worker, nor the host process. The handler may be called concurrently.
func AsyncWithFlushInterval ¶ added in v1.21.0
func AsyncWithFlushInterval(interval time.Duration) AsyncOption
AsyncWithFlushInterval periodically flushes the WRAPPED output - useful for time-buffered inner outputs (e.g. the ElasticSearch bulk output). Zero (the default) disables it. Flush errors are delivered to the error handler.
func AsyncWithPolicy ¶ added in v1.21.0
func AsyncWithPolicy(policy AsyncPolicy) AsyncOption
AsyncWithPolicy sets the full-buffer policy. Default: AsyncPolicyBlock.
type AsyncPolicy ¶ added in v1.21.0
type AsyncPolicy int
AsyncPolicy determines what happens when a message is written to an async output whose buffer is full.
const ( // AsyncPolicyBlock blocks the writer until buffer space is available. // It's the default policy - no message is ever dropped. AsyncPolicyBlock AsyncPolicy = iota // AsyncPolicyDropNewest drops the incoming message. AsyncPolicyDropNewest // AsyncPolicyDropOldest drops the oldest buffered message, making room // for the incoming one. AsyncPolicyDropOldest )
Available policies.
func (AsyncPolicy) String ¶ added in v1.21.0
func (p AsyncPolicy) String() string
String interface implementation.
type ElasticSearchBulkOption ¶ added in v1.21.0
type ElasticSearchBulkOption = elasticsearch.BulkOption
ElasticSearchBulkOption configures the underlying bulk indexer - see the `elasticsearch.BulkWith*` options.
type ElasticSearchConfig ¶ added in v1.6.2
type ElasticSearchConfig = elasticsearch.Config
ElasticSearchConfig is the ElasticSearch configuration.
type ElasticSearchDynamicIndexFunc ¶ added in v1.6.4
type ElasticSearchDynamicIndexFunc = elasticsearch.DynamicIndexFunc
ElasticSearchDynamicIndexFunc is a function which defines the name of the index and is evaluated at the index time.
type ElasticSearchTagMap ¶ added in v1.6.5
type ElasticSearchTagMap = map[string]ElasticSearchTagMapItem
ElasticSearchTagMap is a map of tags to index names.
type ElasticSearchTagMapItem ¶ added in v1.6.5
type ElasticSearchTagMapItem struct {
DynamicIndexFunc ElasticSearchDynamicIndexFunc
Level level.Level
}
ElasticSearchTagMapItem is an item of the ElasticSearch tag map.
func NewElasticSearchTagMapItem ¶ added in v1.6.5
func NewElasticSearchTagMapItem( l level.Level, dynamicIndexFunc ElasticSearchDynamicIndexFunc, ) ElasticSearchTagMapItem
NewElasticSearchTagMapItem is a helper to create `ElasticSearchTagMapItem`.
type IOutput ¶
type IOutput interface {
meta.IMeta
// String interface.
String() string
// GetBuiltinLogger returns the Golang's builtin logger.
GetBuiltinLogger() *builtin.Builtin
// SetBuiltinLogger sets the Golang's builtin logger.
SetBuiltinLogger(builtinLogger *builtin.Builtin) IOutput
// GetFormatter returns the formatter.
GetFormatter() formatter.IFormatter
// SetFormatter sets the formatter.
SetFormatter(fmtr formatter.IFormatter) IOutput
// GetMaxLevel returns the max level.
GetMaxLevel() level.Level
// SetMaxLevel sets the max level.
SetMaxLevel(l level.Level) IOutput
// AddProcessors adds one or more processors.
AddProcessors(processors ...processor.IProcessor) IOutput
// GetProcessor returns the registered processor by its name. If not found, will be nil.
GetProcessor(name string) processor.IProcessor
// SetProcessors sets one or more processors.
SetProcessors(processors ...processor.IProcessor) IOutput
// GetProcessors returns registered processors.
GetProcessors() []processor.IProcessor
// GetProcessorsNames returns the names of the registered processors.
GetProcessorsNames() []string
// GetWriter returns the writer.
GetWriter() io.Writer
// SetWriter sets the writer.
SetWriter(w io.Writer) IOutput
// Write write the message to the defined output.
Write(m message.IMessage) error
}
IOutput specifies what an output does.
func Async ¶ added in v1.21.0
func Async(o IOutput, opts ...AsyncOption) IOutput
Async wraps `o` into a bounded, buffered, asynchronous output: writes enqueue into a buffer (default capacity: 1024), and a single worker goroutine drains it to `o` - preserving FIFO order. All other `IOutput` methods are proxied to `o`, so Sypl-level dispatch (name matching, level checks, status checks) behaves identically.
Capabilities: - `Flush() error`: guarantees everything enqueued BEFORE the call was written to `o` (or dropped by policy), then flushes `o` - if `o` implements `Flush() error`. SNAPSHOT semantics: messages enqueued after the call may remain buffered, so Flush returns even under sustained concurrent writes. - `Close() error`: flushes, stops the worker, and closes `o` - if `o` implements `io.Closer`. Idempotent. Writes after Close return `ErrAsyncClosed`.
Hung sinks: direct Flush, and Close calls wait - UNBOUNDED - on the wrapped output's in-flight write, so a sink that never returns blocks them forever. The Fatal path's pre-exit flush is time-bounded (the process exits regardless), but direct Flush/Close are NOT: prefer bounded sinks (e.g. writers carrying I/O deadlines/timeouts), and `AsyncWithFlushInterval` for time-buffered inner outputs, so draining never depends on a single unbounded call.
See the `Async*` options for buffer size, full-buffer policy, error handling, and periodic flushing.
func Console ¶
func Console(maxLevel level.Level, processors ...processor.IProcessor) IOutput
Console is a built-in `output` - named `Console`, that writes to `stdout`.
func ElasticSearch ¶ added in v1.6.2
func ElasticSearch( indexName string, esConfig ElasticSearchConfig, maxLevel level.Level, processors ...processor.IProcessor, ) IOutput
ElasticSearch is a built-in `output` - named `ElasticSearch`, that writes to ElasticSearch.
NOTE: By default, data is JSON-formatted. NOTE: It's the caller's responsibility to create the index, define its mapping, and settings.
func ElasticSearchBulk ¶ added in v1.21.0
func ElasticSearchBulk( indexName string, esConfig ElasticSearchConfig, maxLevel level.Level, bulkOpts []ElasticSearchBulkOption, processors ...processor.IProcessor, ) IOutput
ElasticSearchBulk is a built-in `output` - named `ElasticSearchBulk`, that writes to ElasticSearch batching documents into _bulk requests - the high-throughput sibling of `ElasticSearch`.
Capabilities: `Flush() error` (drains the indexer), and idempotent `Close() error`. Indexing is asynchronous: per-item failures are delivered through `elasticsearch.BulkWithOnError`.
NOTE: By default, data is JSON-formatted. NOTE: It's the caller's responsibility to create the index, define its mapping, and settings.
func ElasticSearchBulkWithDynamicIndex ¶ added in v1.21.0
func ElasticSearchBulkWithDynamicIndex( dynamicIndexFunc ElasticSearchDynamicIndexFunc, esConfig ElasticSearchConfig, maxLevel level.Level, bulkOpts []ElasticSearchBulkOption, processors ...processor.IProcessor, ) IOutput
ElasticSearchBulkWithDynamicIndex is a built-in `output` - named `ElasticSearchBulkWithDynamicIndex-{index}` that writes to ElasticSearch batching documents into _bulk requests. It allows to define a function that returns the index name to be used, evaluated at the index time.
Capabilities: `Flush() error` (drains the indexer), and idempotent `Close() error`. Indexing is asynchronous: per-item failures are delivered through `elasticsearch.BulkWithOnError`.
NOTE: By default, data is JSON-formatted. NOTE: It's the caller's responsibility to create the index, define its mapping, and settings.
func ElasticSearchWithDynamicIndex ¶ added in v1.6.4
func ElasticSearchWithDynamicIndex( dynamicIndexFunc ElasticSearchDynamicIndexFunc, esConfig ElasticSearchConfig, maxLevel level.Level, processors ...processor.IProcessor, ) IOutput
ElasticSearchWithDynamicIndex is a built-in `output` - named `ElasticSearchWithDynamicIndex` that writes to ElasticSearch. It allows to define a function that returns the index name to be used, evaluated at the index time.
NOTE: By default, data is JSON-formatted. NOTE: It's the caller's responsibility to create the index, define its mapping, and settings.
func ElasticSearchWithTagMap ¶ added in v1.6.5
func ElasticSearchWithTagMap( tagMap ElasticSearchTagMap, esConfig ElasticSearchConfig, processors ...processor.IProcessor, ) []IOutput
ElasticSearchWithTagMap is a built-in `output` - named `ElasticSearchWithTagMap-{tag}` that writes to ElasticSearch. It allows to define a map of tags and indexes. The index name is a function which defines the name of the index and is evaluated at the index time.
IT'S THE CALLER'S RESPONSIBILITY TO DEFINE A CATCH-ALL INDEX - IF DESIRED. TO ACHIEVE THIS, USE `*` AS THE TAG NAME.
NOTE: By default, data is JSON-formatted. NOTE: It's the caller's responsibility to create the index, define its mapping, and settings.
func File ¶
func File(name string, path string, maxLevel level.Level, processors ...processor.IProcessor) IOutput
File is a built-in `output` - named `File`, that writes to the specified file.
NOTE: If the common used "-" is used, it will behave as a Console writing to stdout. NOTE: If no path is provided, it'll create one in the OS's temp directory. NOTE: If the dir and/or file does not exist, it will be created.
func FileBased ¶
func FileBased( name string, maxLevel level.Level, writer io.Writer, processors ...processor.IProcessor, ) IOutput
FileBased is a built-in `output`, that writes to the specified file.
func New ¶
func New(name string, maxLevel level.Level, w io.Writer, processors ...processor.IProcessor, ) IOutput
New is the Output factory.
func RotatingFile ¶ added in v1.21.0
func RotatingFile( name, path string, maxLevel level.Level, cfg RotationConfig, processors ...processor.IProcessor, ) (IOutput, error)
RotatingFile is a built-in `output` that writes to the specified file, rotating it by size: when a write would push the live file beyond `cfg.MaxSizeBytes`, the file is closed, renamed to `<path>.<UTC timestamp>`, and reopened fresh - then backups beyond `cfg.MaxBackups`, or older than `cfg.MaxAgeDays`, are pruned (inline, no goroutines).
Capabilities: `Flush() error` (file sync), and idempotent `Close() error`. Writes after Close return `ErrRotatingFileClosed`.
Notes: - Unlike `File`, it returns an error - it never calls log.Fatalf. - Missing parent directories are created. - Any file named `<path>.*` is treated as a backup by pruning. - SELF-HEALING: a mid-rotation failure (e.g. the rename was denied) reopens the original path, so writes keep landing there - the failure is surfaced through the write's error, and rotation retries on the next size-threshold write. When even the reopen fails, writes return `ErrRotatingFileUnavailable` - retrying the reopen each time - until the filesystem recovers.
func SafeBuffer ¶
func SafeBuffer(maxLevel level.Level, processors ...processor.IProcessor) (*safebuffer.Buffer, IOutput)
SafeBuffer is a built-in `output` - named `Buffer`, that writes to the buffer.
func StdErr ¶
func StdErr(processors ...processor.IProcessor) IOutput
StdErr is a built-in `output` - named `StdErr`, that only writes to `stderr` message @ Error level.
type Record ¶ added in v1.21.0
type Record struct {
// Fields is a copy of the message's structured fields.
Fields fields.Fields
// Level is the message's level.
Level level.Level
// OriginalContent is the message's content before processing.
OriginalContent string
// OutputName is the output the message was routed to.
OutputName string
// ProcessedContent is the exact text written - processors, and
// formatter applied.
ProcessedContent string
// ProcessorsNames is a copy of the processors names applied.
ProcessorsNames []string
// Tags is a copy of the message's tags - lexicographically sorted.
Tags []string
// Timestamp is the message's creation time.
Timestamp time.Time
}
Record is a structured snapshot of a message the recorder output actually wrote - level gating, flags, and processors all applied.
type RecorderOutput ¶ added in v1.21.0
type RecorderOutput struct {
// contains filtered or unexported fields
}
RecorderOutput is an `IOutput` capturing a structured snapshot of every message it writes - a test-assertion helper for Sypl consumers.
func (RecorderOutput) AddProcessors ¶ added in v1.21.0
func (p RecorderOutput) AddProcessors(processors ...processor.IProcessor) IOutput
AddProcessors adds one or more processors to the inner output.
func (RecorderOutput) GetBuiltinLogger ¶ added in v1.21.0
GetBuiltinLogger returns the inner output's Golang's builtin logger.
func (RecorderOutput) GetFormatter ¶ added in v1.21.0
func (p RecorderOutput) GetFormatter() formatter.IFormatter
GetFormatter returns the inner output's formatter.
func (RecorderOutput) GetMaxLevel ¶ added in v1.21.0
GetMaxLevel returns the inner output's max level.
func (RecorderOutput) GetName ¶ added in v1.21.0
func (p RecorderOutput) GetName() string
GetName returns the inner output name.
func (RecorderOutput) GetProcessor ¶ added in v1.21.0
func (p RecorderOutput) GetProcessor(name string) processor.IProcessor
GetProcessor returns the inner output's registered processor by its name. If not found, will be nil.
func (RecorderOutput) GetProcessors ¶ added in v1.21.0
func (p RecorderOutput) GetProcessors() []processor.IProcessor
GetProcessors returns the inner output's registered processors.
func (RecorderOutput) GetProcessorsNames ¶ added in v1.21.0
func (p RecorderOutput) GetProcessorsNames() []string
GetProcessorsNames returns the names of the inner output's registered processors.
func (*RecorderOutput) Len ¶ added in v1.21.0
func (r *RecorderOutput) Len() int
Len returns how many records were captured.
func (*RecorderOutput) Messages ¶ added in v1.21.0
func (r *RecorderOutput) Messages() []Record
Messages returns a defensive copy of the captured records.
func (*RecorderOutput) Reset ¶ added in v1.21.0
func (r *RecorderOutput) Reset()
Reset discards the captured records.
func (RecorderOutput) SetBuiltinLogger ¶ added in v1.21.0
SetBuiltinLogger sets the inner output's Golang's builtin logger.
func (RecorderOutput) SetFormatter ¶ added in v1.21.0
func (p RecorderOutput) SetFormatter(fmtr formatter.IFormatter) IOutput
SetFormatter sets the inner output's formatter.
func (RecorderOutput) SetMaxLevel ¶ added in v1.21.0
SetMaxLevel sets the inner output's max level.
func (RecorderOutput) SetName ¶ added in v1.21.0
func (p RecorderOutput) SetName(name string)
SetName sets the inner output name.
func (RecorderOutput) SetProcessors ¶ added in v1.21.0
func (p RecorderOutput) SetProcessors(processors ...processor.IProcessor) IOutput
SetProcessors sets one or more processors on the inner output.
type RotationConfig ¶ added in v1.21.0
type RotationConfig struct {
// MaxSizeBytes is the size threshold: a write that would push the live
// file BEYOND it triggers a rotation first. A write landing exactly at
// the limit does not rotate. Must be positive.
MaxSizeBytes int64
// MaxBackups caps how many rotated backups are kept - the oldest
// beyond the cap are pruned on rotation. Zero keeps all.
MaxBackups int
// MaxAgeDays prunes - on rotation - backups whose modification time is
// older than this many days. Zero keeps all.
MaxAgeDays int
}
RotationConfig configures the rotating file output.