Documentation
¶
Overview ¶
Package reader provides a unified and advanced reader interface for handling text from both files and streams.
It is designed to abstract away the complexities of ingestion, stream processing, and initial text decoration. The primary features of this package include:
- Input Unification: Provides a clean `Reader` interface to query lines (`GetLines`, `GetLine`), regardless of whether the source is a static file or a continuous stream.
- Transparent Decompression: Automatically detects and decompresses formats like gzip, bzip2, zstd, and xz on-the-fly. This is handled by the ZOpen function, which inspects magic bytes and strips compression extensions.
- Content Format Detection and Reformatting: Checks the incoming text for valid JSON or XML if highlighting information isn't provided, and optionally pretty-prints JSON when requested via ReaderOptions.ShouldFormat.
- Live Tailing: Continues to monitor seekable file sources (using `tailFile`) for appended bytes or truncated reloads, updating the viewer automatically.
- Syntax Highlighting: Applies highlighting using Chroma (alecthomas/chroma). Reading and highlighting happen asynchronously in the background so the UI doesn't block.
- Pausing and Resource Management: Prevents infinite memory consumption by intelligently pausing reads at a specific line limit (e.g. 50,000 lines) until the user scrolls further down.
- Status and Metadata Generation: Generates view-ready status lines indicating buffer name, current view position, and trailing progress percentages as the data dictates (`createStatusUnlocked`).
By encapsulating these capabilities, the reader package allows the rest of the application (such as the pager) to consume rich, formatted, lines of text without worrying about the underlying complexities of text ingestion.
Index ¶
- Constants
- Variables
- func Highlight(text string, style chroma.Style, formatter chroma.Formatter, ...) (*string, error)
- func PanicHandler(goroutineName string, recoverResult any, stackTrace []byte)
- func TryOpen(filename string) error
- func ZOpen(filename string) (io.ReadCloser, string, error)
- func ZReader(input io.Reader) (io.Reader, error)
- type InputLines
- type Line
- type NumberedLine
- type Reader
- type ReaderImpl
- func (reader *ReaderImpl) AwaitFirstByte()
- func (reader *ReaderImpl) Clone() (*ReaderImpl, error)
- func (reader *ReaderImpl) Close()
- func (reader *ReaderImpl) GetLine(index linemetadata.Index) *NumberedLine
- func (reader *ReaderImpl) GetLineCount() int
- func (reader *ReaderImpl) GetLines(firstLine linemetadata.Index, wantedLineCount int) InputLines
- func (reader *ReaderImpl) GetLinesPreallocated(firstLine linemetadata.Index, resultLines *[]NumberedLine) Status
- func (reader *ReaderImpl) PumpToStdout()
- func (reader *ReaderImpl) SetPauseAfterLines(lines int)
- func (reader *ReaderImpl) SetStyleForHighlighting(style chroma.Style)
- func (reader *ReaderImpl) ShouldShowLineCount() bool
- func (reader *ReaderImpl) Wait() error
- type ReaderOptions
- type Status
Constants ¶
const DEFAULT_PAUSE_AFTER_LINES = 50_000
To cap resource usage when not needed, start by reading this many lines into memory. If the user scrolls near the end or starts searching, we'll read more.
Ref: https://github.com/walles/moor/issues/296
const MAX_HIGHLIGHT_SIZE int64 = 2_000_000
An 1.7MB file took 2s to highlight. The number for this limit is totally negotiable.
Variables ¶
var DisablePlainCachingForBenchmarking = false
Functions ¶
func Highlight ¶
func Highlight(text string, style chroma.Style, formatter chroma.Formatter, lexer chroma.Lexer) (*string, error)
Read and highlight some text using Chroma: https://github.com/alecthomas/chroma
If lexer is nil no highlighting will be performed.
Returns nil with no error if highlighting would be a no-op, which includes the case where every visible character would end up looking the same.
func PanicHandler ¶
func ZOpen ¶
func ZOpen(filename string) (io.ReadCloser, string, error)
The second return value is the file name with any compression extension removed.
Types ¶
type InputLines ¶
type InputLines struct {
Lines []NumberedLine
Status
}
InputLines contains a number of lines from the reader, plus metadata
type Line ¶
type Line struct {
// contains filtered or unexported fields
}
func (*Line) HasManPageFormatting ¶ added in v2.9.1
func (*Line) HighlightedTokens ¶
func (line *Line) HighlightedTokens( plainTextStyle twin.Style, searchHitStyle twin.Style, activeSearch search.Search, lineIndex linemetadata.Index, maxCellsCount int, ) textstyles.StyledRunesWithTrailer
Returns a representation of the string split into styled tokens. Any regexp matches are highlighted. A nil regexp means no highlighting.
maxCellsCount: at most this many cells will be included in the result. If 0, there is no limit. For BenchmarkRenderHugeLine() performance.
func (*Line) IsManPageTitleLine ¶ added in v2.19.6
type NumberedLine ¶
type NumberedLine struct {
Index linemetadata.Index
Number linemetadata.Number
Line *Line
}
func (*NumberedLine) DisplayWidth ¶ added in v2.5.0
func (nl *NumberedLine) DisplayWidth() int
func (*NumberedLine) HighlightedTokens ¶
func (nl *NumberedLine) HighlightedTokens(plainTextStyle twin.Style, searchHitStyle twin.Style, search search.Search, maxCellsCount int) textstyles.StyledRunesWithTrailer
maxCellsCount: at most this many cells will be included in the result. If 0, there is no limit. For BenchmarkRenderHugeLine() performance.
func (*NumberedLine) Plain ¶
func (nl *NumberedLine) Plain() string
type Reader ¶
type Reader interface {
GetLineCount() int
GetLine(index linemetadata.Index) *NumberedLine
// This method will try to honor wantedLineCount over firstLine. This means
// that the returned first line may be different from the requested one.
GetLines(firstLine linemetadata.Index, wantedLineCount int) InputLines
// GetLines gets the indicated lines from the input. The lines will be stored
// in the provided preallocated slice to avoid allocations. The line count is
// determined by the capacity of the provided slice.
//
// The return value is the status for the returned lines.
GetLinesPreallocated(firstLine linemetadata.Index, resultLines *[]NumberedLine) Status
// False when paused. Showing the paused line count is confusing, because
// the user might think that the number is the total line count, even though
// we are not done yet.
//
// When we're not paused, the number will be constantly changing, indicating
// that the counting is not done yet.
ShouldShowLineCount() bool
}
type ReaderImpl ¶
type ReaderImpl struct {
sync.RWMutex
// Display name for the buffer. If not set, no buffer name will be shown.
//
// For files, this will be the basename of the file. For our help text, this
// will be "Help". For streams this will generally not be set, but may come
// from the $PAGER_LABEL environment variable.
DisplayName *string
// If this is set, it will point out the file we are reading from. If this
// is not set, we are not reading from a file.
FileName *string
// True if the file we read from was compressed.
IsCompressed bool
Err error
// When we started reading this stream, once its first bytes had already
// arrived. Nothing paints before that point (see AwaitFirstByte(), which
// blocks pager startup on exactly this), so time spent waiting for a slow
// producer's first byte can't cause a --quit-if-one-screen blink and is
// correctly excluded here. Used for reporting how far into the read we were
// when something happened, alongside the "Stream read in" duration logged
// once ReadingDone.
StartedAt time.Time
// Stream has been completely read. May not be highlighted yet.
ReadingDone *atomic.Bool
// Highlighting has been completed.
HighlightingDone *atomic.Bool
// For telling the UI it should recheck the --quit-if-one-screen conditions.
// Signalled when either highlighting is done or reading is done.
MaybeDone chan bool
MoreLinesAdded chan bool
// PauseStatus is true if the reader is paused, false if it is not
PauseStatus *atomic.Bool
// contains filtered or unexported fields
}
ReaderImpl reads a file into an array of strings.
It does the reading in the background, and it returns parts of the read data upon request.
This package provides query methods for the struct, no peeking!!
func NewFromFilename ¶
func NewFromFilename(filename string, formatter chroma.Formatter, options ReaderOptions) (*ReaderImpl, error)
NewFromFilename creates a new file reader.
If options.Lexer is nil it will be determined from the input file name.
If options.Style is nil, you must call reader.SetStyleForHighlighting() later to get highlighting.
The Reader will try to uncompress various compressed file format, and also apply highlighting to the file using Chroma: https://github.com/alecthomas/chroma
func NewFromStream ¶
func NewFromStream(displayName string, reader io.Reader, formatter chroma.Formatter, options ReaderOptions) (*ReaderImpl, error)
NewFromStream creates a new stream reader
The reader takes ownership of the stream: if it implements io.Closer it will be closed once we have consumed its initial content, or on reader.Close(), whichever comes first.
The display name can be an empty string ("").
If non-empty, the name will be displayed by the pager in the bottom left corner to help the user keep track of what is being paged.
Note that you must call reader.SetStyleForHighlighting() after this to get highlighting.
func NewFromTextForTesting ¶
func NewFromTextForTesting(name string, text string) *ReaderImpl
Testing only!! May or may not hang if run in real world scenarios.
NewFromTextForTesting creates a Reader from a block of text.
First parameter is the name of this Reader. This name will be displayed by Moor in the bottom left corner of the screen.
Calling Wait() on this Reader will always return immediately, no asynchronous ops will be performed.
func (*ReaderImpl) AwaitFirstByte ¶
func (reader *ReaderImpl) AwaitFirstByte()
Wait for the first line to be read.
Used for making sudo work: https://github.com/walles/moor/issues/199
func (*ReaderImpl) Clone ¶ added in v2.13.0
func (reader *ReaderImpl) Clone() (*ReaderImpl, error)
Clone creates a new ReaderImpl using the same source file and options.
func (*ReaderImpl) Close ¶ added in v2.13.0
func (reader *ReaderImpl) Close()
Close stops background routines and prevents further reading.
The stream we were created from gets closed, which is how a producer still writing into it finds out that nobody is listening any more.
func (*ReaderImpl) GetLine ¶
func (reader *ReaderImpl) GetLine(index linemetadata.Index) *NumberedLine
GetLine gets a line. If the requested line number is out of bounds, nil is returned.
func (*ReaderImpl) GetLineCount ¶
func (reader *ReaderImpl) GetLineCount() int
GetLineCount returns the number of lines available for viewing
func (*ReaderImpl) GetLines ¶
func (reader *ReaderImpl) GetLines(firstLine linemetadata.Index, wantedLineCount int) InputLines
GetLines gets the indicated lines from the input
func (*ReaderImpl) GetLinesPreallocated ¶ added in v2.9.3
func (reader *ReaderImpl) GetLinesPreallocated(firstLine linemetadata.Index, resultLines *[]NumberedLine) Status
GetLines gets the indicated lines from the input. The lines will be stored in the provided preallocated slice to avoid allocations. The line count is determined by the capacity of the provided slice.
The return value is the status for the returned lines.
func (*ReaderImpl) PumpToStdout ¶
func (reader *ReaderImpl) PumpToStdout()
func (*ReaderImpl) SetPauseAfterLines ¶
func (reader *ReaderImpl) SetPauseAfterLines(lines int)
func (*ReaderImpl) SetStyleForHighlighting ¶
func (reader *ReaderImpl) SetStyleForHighlighting(style chroma.Style)
func (*ReaderImpl) ShouldShowLineCount ¶
func (reader *ReaderImpl) ShouldShowLineCount() bool
func (*ReaderImpl) Wait ¶
func (reader *ReaderImpl) Wait() error
Wait for reader to finish reading and highlighting. Used by tests.
type ReaderOptions ¶
type ReaderOptions struct {
// Format JSON input
ShouldFormat bool
// Pause after reading this many lines, unless told otherwise.
// Tune at runtime using SetPauseAfterLines().
//
// nil means 20k lines.
PauseAfterLines *int
// If this is nil, you must call reader.SetStyleForHighlighting() later if
// you want highlighting.
Style *chroma.Style
// If this is set, it will be used as the lexer for highlighting
Lexer chroma.Lexer
}