reader

package
v2.19.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 14, 2026 License: BSD-2-Clause-Views Imports: 31 Imported by: 0

Documentation

Overview

Package reader provides a unified and advanced reader interface for handling text from both files and streams.

It is designed to abstract away the complexities of ingestion, stream processing, and initial text decoration. The primary features of this package include:

  1. Input Unification: Provides a clean `Reader` interface to query lines (`GetLines`, `GetLine`), regardless of whether the source is a static file or a continuous stream.
  2. Transparent Decompression: Automatically detects and decompresses formats like gzip, bzip2, zstd, and xz on-the-fly. This is handled by the ZOpen function, which inspects magic bytes and strips compression extensions.
  3. Content Format Detection and Reformatting: Checks the incoming text for valid JSON or XML if highlighting information isn't provided, and optionally pretty-prints JSON when requested via ReaderOptions.ShouldFormat.
  4. Live Tailing: Continues to monitor seekable file sources (using `tailFile`) for appended bytes or truncated reloads, updating the viewer automatically.
  5. Syntax Highlighting: Applies highlighting using Chroma (alecthomas/chroma). Reading and highlighting happen asynchronously in the background so the UI doesn't block.
  6. Pausing and Resource Management: Prevents infinite memory consumption by intelligently pausing reads at a specific line limit (e.g. 50,000 lines) until the user scrolls further down.
  7. Status and Metadata Generation: Generates view-ready status lines indicating buffer name, current view position, and trailing progress percentages as the data dictates (`createStatusUnlocked`).

By encapsulating these capabilities, the reader package allows the rest of the application (such as the pager) to consume rich, formatted, lines of text without worrying about the underlying complexities of text ingestion.

Index

Constants

View Source
const DEFAULT_PAUSE_AFTER_LINES = 50_000

To cap resource usage when not needed, start by reading this many lines into memory. If the user scrolls near the end or starts searching, we'll read more.

Ref: https://github.com/walles/moor/issues/296

View Source
const MAX_HIGHLIGHT_SIZE int64 = 2_000_000

An 1.7MB file took 2s to highlight. The number for this limit is totally negotiable.

Variables

View Source
var DisablePlainCachingForBenchmarking = false

Functions

func Highlight

func Highlight(text string, style chroma.Style, formatter chroma.Formatter, lexer chroma.Lexer) (*string, error)

Read and highlight some text using Chroma: https://github.com/alecthomas/chroma

If lexer is nil no highlighting will be performed.

Returns nil with no error if highlighting would be a no-op, which includes the case where every visible character would end up looking the same.

func PanicHandler

func PanicHandler(goroutineName string, recoverResult any, stackTrace []byte)

func TryOpen

func TryOpen(filename string) error

func ZOpen

func ZOpen(filename string) (io.ReadCloser, string, error)

The second return value is the file name with any compression extension removed.

func ZReader

func ZReader(input io.Reader) (io.Reader, error)

ZReader returns a reader that decompresses the input stream. Any input stream compression will be automatically detected. Uncompressed streams will be returned as-is.

If the input is an io.Closer then so is the returned reader, and closing that closes the input.

Ref: https://github.com/walles/moor/issues/261

Types

type InputLines

type InputLines struct {
	Lines []NumberedLine
	Status
}

InputLines contains a number of lines from the reader, plus metadata

type Line

type Line struct {
	// contains filtered or unexported fields
}

func (*Line) HasManPageFormatting added in v2.9.1

func (line *Line) HasManPageFormatting() bool

func (*Line) HighlightedTokens

func (line *Line) HighlightedTokens(
	plainTextStyle twin.Style,
	searchHitStyle twin.Style,
	activeSearch search.Search,
	lineIndex linemetadata.Index,
	maxCellsCount int,
) textstyles.StyledRunesWithTrailer

Returns a representation of the string split into styled tokens. Any regexp matches are highlighted. A nil regexp means no highlighting.

maxCellsCount: at most this many cells will be included in the result. If 0, there is no limit. For BenchmarkRenderHugeLine() performance.

func (*Line) IsManPageTitleLine added in v2.19.6

func (line *Line) IsManPageTitleLine() bool

func (*Line) Plain

func (line *Line) Plain(index linemetadata.Index) string

The index is for error reporting. Set DisablePlainCachingForBenchmarking to false to simulate a cache miss for benchmarking.

type NumberedLine

type NumberedLine struct {
	Index  linemetadata.Index
	Number linemetadata.Number
	Line   *Line
}

func (*NumberedLine) DisplayWidth added in v2.5.0

func (nl *NumberedLine) DisplayWidth() int

func (*NumberedLine) HighlightedTokens

func (nl *NumberedLine) HighlightedTokens(plainTextStyle twin.Style, searchHitStyle twin.Style, search search.Search, maxCellsCount int) textstyles.StyledRunesWithTrailer

maxCellsCount: at most this many cells will be included in the result. If 0, there is no limit. For BenchmarkRenderHugeLine() performance.

func (*NumberedLine) Plain

func (nl *NumberedLine) Plain() string

type Reader

type Reader interface {
	GetLineCount() int
	GetLine(index linemetadata.Index) *NumberedLine

	// This method will try to honor wantedLineCount over firstLine. This means
	// that the returned first line may be different from the requested one.
	GetLines(firstLine linemetadata.Index, wantedLineCount int) InputLines

	// GetLines gets the indicated lines from the input. The lines will be stored
	// in the provided preallocated slice to avoid allocations. The line count is
	// determined by the capacity of the provided slice.
	//
	// The return value is the status for the returned lines.
	GetLinesPreallocated(firstLine linemetadata.Index, resultLines *[]NumberedLine) Status

	// False when paused. Showing the paused line count is confusing, because
	// the user might think that the number is the total line count, even though
	// we are not done yet.
	//
	// When we're not paused, the number will be constantly changing, indicating
	// that the counting is not done yet.
	ShouldShowLineCount() bool
}

type ReaderImpl

type ReaderImpl struct {
	sync.RWMutex

	// Display name for the buffer. If not set, no buffer name will be shown.
	//
	// For files, this will be the basename of the file. For our help text, this
	// will be "Help". For streams this will generally not be set, but may come
	// from the $PAGER_LABEL environment variable.
	DisplayName *string

	// If this is set, it will point out the file we are reading from. If this
	// is not set, we are not reading from a file.
	FileName *string

	// True if the file we read from was compressed.
	IsCompressed bool

	Err error

	// When we started reading this stream, once its first bytes had already
	// arrived. Nothing paints before that point (see AwaitFirstByte(), which
	// blocks pager startup on exactly this), so time spent waiting for a slow
	// producer's first byte can't cause a --quit-if-one-screen blink and is
	// correctly excluded here. Used for reporting how far into the read we were
	// when something happened, alongside the "Stream read in" duration logged
	// once ReadingDone.
	StartedAt time.Time

	// Stream has been completely read. May not be highlighted yet.
	ReadingDone *atomic.Bool

	// Highlighting has been completed.
	HighlightingDone *atomic.Bool

	// For telling the UI it should recheck the --quit-if-one-screen conditions.
	// Signalled when either highlighting is done or reading is done.
	MaybeDone chan bool

	MoreLinesAdded chan bool

	// PauseStatus is true if the reader is paused, false if it is not
	PauseStatus *atomic.Bool
	// contains filtered or unexported fields
}

ReaderImpl reads a file into an array of strings.

It does the reading in the background, and it returns parts of the read data upon request.

This package provides query methods for the struct, no peeking!!

func NewFromFilename

func NewFromFilename(filename string, formatter chroma.Formatter, options ReaderOptions) (*ReaderImpl, error)

NewFromFilename creates a new file reader.

If options.Lexer is nil it will be determined from the input file name.

If options.Style is nil, you must call reader.SetStyleForHighlighting() later to get highlighting.

The Reader will try to uncompress various compressed file format, and also apply highlighting to the file using Chroma: https://github.com/alecthomas/chroma

func NewFromStream

func NewFromStream(displayName string, reader io.Reader, formatter chroma.Formatter, options ReaderOptions) (*ReaderImpl, error)

NewFromStream creates a new stream reader

The reader takes ownership of the stream: if it implements io.Closer it will be closed once we have consumed its initial content, or on reader.Close(), whichever comes first.

The display name can be an empty string ("").

If non-empty, the name will be displayed by the pager in the bottom left corner to help the user keep track of what is being paged.

Note that you must call reader.SetStyleForHighlighting() after this to get highlighting.

func NewFromTextForTesting

func NewFromTextForTesting(name string, text string) *ReaderImpl

Testing only!! May or may not hang if run in real world scenarios.

NewFromTextForTesting creates a Reader from a block of text.

First parameter is the name of this Reader. This name will be displayed by Moor in the bottom left corner of the screen.

Calling Wait() on this Reader will always return immediately, no asynchronous ops will be performed.

func (*ReaderImpl) AwaitFirstByte

func (reader *ReaderImpl) AwaitFirstByte()

Wait for the first line to be read.

Used for making sudo work: https://github.com/walles/moor/issues/199

func (*ReaderImpl) Clone added in v2.13.0

func (reader *ReaderImpl) Clone() (*ReaderImpl, error)

Clone creates a new ReaderImpl using the same source file and options.

func (*ReaderImpl) Close added in v2.13.0

func (reader *ReaderImpl) Close()

Close stops background routines and prevents further reading.

The stream we were created from gets closed, which is how a producer still writing into it finds out that nobody is listening any more.

func (*ReaderImpl) GetLine

func (reader *ReaderImpl) GetLine(index linemetadata.Index) *NumberedLine

GetLine gets a line. If the requested line number is out of bounds, nil is returned.

func (*ReaderImpl) GetLineCount

func (reader *ReaderImpl) GetLineCount() int

GetLineCount returns the number of lines available for viewing

func (*ReaderImpl) GetLines

func (reader *ReaderImpl) GetLines(firstLine linemetadata.Index, wantedLineCount int) InputLines

GetLines gets the indicated lines from the input

func (*ReaderImpl) GetLinesPreallocated added in v2.9.3

func (reader *ReaderImpl) GetLinesPreallocated(firstLine linemetadata.Index, resultLines *[]NumberedLine) Status

GetLines gets the indicated lines from the input. The lines will be stored in the provided preallocated slice to avoid allocations. The line count is determined by the capacity of the provided slice.

The return value is the status for the returned lines.

func (*ReaderImpl) PumpToStdout

func (reader *ReaderImpl) PumpToStdout()

func (*ReaderImpl) SetPauseAfterLines

func (reader *ReaderImpl) SetPauseAfterLines(lines int)

func (*ReaderImpl) SetStyleForHighlighting

func (reader *ReaderImpl) SetStyleForHighlighting(style chroma.Style)

func (*ReaderImpl) ShouldShowLineCount

func (reader *ReaderImpl) ShouldShowLineCount() bool

func (*ReaderImpl) Wait

func (reader *ReaderImpl) Wait() error

Wait for reader to finish reading and highlighting. Used by tests.

type ReaderOptions

type ReaderOptions struct {
	// Format JSON input
	ShouldFormat bool

	// Pause after reading this many lines, unless told otherwise.
	// Tune at runtime using SetPauseAfterLines().
	//
	// nil means 20k lines.
	PauseAfterLines *int

	// If this is nil, you must call reader.SetStyleForHighlighting() later if
	// you want highlighting.
	Style *chroma.Style

	// If this is set, it will be used as the lexer for highlighting
	Lexer chroma.Lexer
}

type Status added in v2.17.0

type Status struct {
	// Example: "monkey.txt".
	FilenameText string

	// Example: ": 45 lines  51%"
	StatusText string
}

Status describes the current view into the input, for display in the status bar and reporting to the terminal.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址