streams

package
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 10, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var Goroutinewg *sync.WaitGroup

Functions

func MakeCommands

func MakeCommands() commands

func Parse

func Parse(filename string, b []byte, opts ...Option) (interface{}, error)

Parse parses the data from b using filename as information in the error messages.

func ParseFile

func ParseFile(filename string, opts ...Option) (i interface{}, err error)

ParseFile parses the file identified by filename.

func ParseReader

func ParseReader(filename string, r io.Reader, opts ...Option) (interface{}, error)

ParseReader parses the data from r using filename as information in the error messages.

func Tokens added in v1.2.0

func Tokens() []string

Tokens is the list offered for completion and named in error messages.

Types

type Bytes

type Bytes struct {
	Dirn Direction
	Data []byte
}

func (Bytes) Direction

func (b Bytes) Direction() Direction

func (Bytes) StreamData

func (b Bytes) StreamData() []byte

func (Bytes) String

func (b Bytes) String() string

type Cloner

type Cloner interface {
	Clone() interface{}
}

Cloner is implemented by any value that has a Clone method, which returns a copy of the value. This is mainly used for types which are not passed by value (e.g map, slice, chan) or structs that contain such types.

This is used in conjunction with the global state feature to create proper copies of the state to allow the parser to properly restore the state in the case of backtracking.

type Direction

type Direction int
const (
	Client Direction = 0
	Server Direction = iota
)

func (Direction) String

func (d Direction) String() string

type Family added in v1.2.0

type Family struct {
	Proto Protocol

	// Token is the spelling tshark wants in -z follow,<token>,raw,<idx>. It is
	// also the spelling the parser has to accept in the "Follow:" header line.
	Token string

	// Label is what the user is told - it appears in the view's title.
	Label string

	// IndexField is the display filter field that carries the stream index.
	// It is not always the family's own name: a WebSocket stream is indexed by
	// the TCP stream underneath it, and tshark itself emits `tcp.stream eq N`
	// as the filter for follow,websocket. There is no websocket.stream field
	// at all - asking tshark for one is an error.
	IndexField string

	// Transport is the family whose length field says whether a packet carried
	// payload, which is how the indexer decides which packets a chunk can jump
	// to. For everything layered over TCP that is tcp.
	Transport string

	// Layer is the PDML <proto name=...> that must be present for this family
	// to apply to a packet. For tcp and udp it is the same as Token; for the
	// families layered on top it is what distinguishes them from plain TCP.
	Layer string

	// EmptyReason, when set, is why this family can legitimately come back
	// with nothing for a stream that plainly exists - said to the user instead
	// of leaving them with an empty pane.
	//
	// This is not a hypothetical. Measured on a synthesised TLS capture:
	// `-z follow,tls,raw,0` with no key log answers with a complete banner,
	// `Node 0: :0`, no payload and exit status 0 - byte for byte the shape of
	// "there is no such stream" - while `-z follow,tcp,raw,0` over the same
	// stream returns the bytes.
	EmptyReason string

	// Explicit families are not chosen for the user. See PickFor.
	Explicit bool
}

A Family is one of tshark's -z follow,<family> taps, together with the four things this program has to know to drive it.

Measured against tshark 4.6.8, which offers thirteen of them. Four are here; the rest are left out for reasons written down beside them below.

func Families added in v1.2.0

func Families() []Family

Families layered over a transport are asked for, never guessed at. A packet carrying TLS is also a TCP packet, and choosing TLS for it would replace the bytes the user used to get with an empty pane whenever there is no key log - which is the common case, and which tshark reports as success.

Deliberately absent, with the measurement that rules each one out:

  • http2 and quic need TWO indexes - `follow,http2,raw,<tcp stream>,<http2 streamid>`. One index is refused: "Error creating filter for this stream", exit 1. The loader's Stream/Indexer/StartLoad take a single int, and follow.peg's FollowExpr is [a-zA-Z,]+, which rejects the digit in a `Follow: http2,raw` header - verified by feeding one to ParseReader.
  • mp2t and mpeg-pes need two indexes as well, spell their filter with ==/&& rather than eq/and, and give the PID in decimal while printing it as hex.
  • sip is registered by tshark and unusable from the command line: every index form, and the Call-ID form, is refused with exit 1.
  • http reverses the node order relative to follow,tcp on the same stream and returns the reassembled body as a single untabbed chunk, losing the request - it is a worse answer than the TCP stream, not a better one.
  • dccp, dtls and usbcom are transports this program has no captures of and no way to make one on this machine.

func FamilyByToken added in v1.2.0

func FamilyByToken(tok string) (Family, bool)

FamilyByToken looks up what the user typed after :streams.

func FamilyOf added in v1.2.0

func FamilyOf(p Protocol) Family

FamilyOf returns the family for a Protocol. Every value of the enum has one; a value that does not is a programming error and says so.

func (Family) Filter added in v1.2.0

func (f Family) Filter(idx int) string

Filter is the display filter for one stream of this family. It is the same expression tshark prints in the Filter: line of its own output, which is what makes the packet list and the stream view agree about what is being shown.

func (Family) Resolve added in v1.2.0

func (f Family) Resolve(known func(string) bool) Family

Resolve adjusts the index field to the fields this tshark actually has.

tls.stream does not exist before Wireshark 4.4. The TLS follow tap is there in 4.2 and reports `tcp.stream eq N` as its own filter - measured on the CI runner's 4.2.2, against 4.6.8 here, which reports tls.stream. That matters beyond a label: a display filter naming a field this build does not know is rejected outright, so the index pass would fail rather than come back empty, and following a TLS stream would break on a Wireshark this program is supposed to support.

known answers whether a field name exists; nil means "assume it does", which is what happens before the field list has loaded.

type FollowHeader

type FollowHeader struct {
	Follow string
	Filter string
	Node0  string
	Node1  string
}

func (FollowHeader) Clean added in v1.2.0

func (f FollowHeader) Clean() FollowHeader

Clean returns the header with the line endings taken off its four fields.

tshark writes CRLF on Windows and the grammar captures a header line up to the newline, so every field arrives with a carriage return on the end there and without one everywhere else. Node0 and Node1 are drawn into the conversation menu and into the "client → server (N bytes)" line, where a stray CR lands in the middle of the text and sends the cursor back to the start of the row.

func (FollowHeader) String

func (h FollowHeader) String() string

type FollowStream

type FollowStream struct {
	FollowHeader
	Bytes []Bytes
}

func (FollowStream) String

func (f FollowStream) String() string

type IChunk

type IChunk interface {
	Direction() Direction
	StreamData() []byte
}

type IIndexerCallbacks

type IIndexerCallbacks interface {
	IOnStreamChunk
	ITrackPayload
	AfterIndexEnd(success bool)
}

type ILoaderCmds

type ILoaderCmds interface {
	// token is the tshark spelling of the family - tcp, udp, tls, websocket.
	Stream(pcap string, token string, idx int) pcap.IPcapCommand
	// The indexer is given the whole display filter rather than a family and
	// an index, because the field carrying the index is not always named
	// after the family: a WebSocket stream is indexed by tcp.stream.
	Indexer(pcap string, filter string) pcap.IPcapCommand
}

type IOnStreamChunk

type IOnStreamChunk interface {
	OnStreamChunk(chunk IChunk)
}

type IOnStreamHeader

type IOnStreamHeader interface {
	OnStreamHeader(header FollowHeader)
}

type ISavedData

type ISavedData interface {
	NumChunks() int
	Chunk(i int) IChunk
}

type ITrackPayload

type ITrackPayload interface {
	TrackPayloadPacket(packet int)
}

type Loader

type Loader struct {
	SuppressErrors bool // if true, don't report process errors e.g. at shutdown
	// contains filtered or unexported fields
}

func NewLoader

func NewLoader(cmds ILoaderCmds, ctx context.Context) *Loader

func (*Loader) StartLoad

func (c *Loader) StartLoad(pcap string, f Family, idx int, app gowid.IApp, cb IIndexerCallbacks)

func (*Loader) StopLoad

func (c *Loader) StopLoad()

Called by user to cancel a stream reassembly op. Stops both processes straight away. Note that typically, the indexer will be further behind.

type Option

type Option func(*parser) Option

Option is a function that can set an option on the parser. It returns the previous setting as an Option.

func AllowInvalidUTF8

func AllowInvalidUTF8(b bool) Option

AllowInvalidUTF8 creates an Option to allow invalid UTF-8 bytes. Every invalid UTF-8 byte is treated as a utf8.RuneError (U+FFFD) by character class matchers and is matched by the any matcher. The returned matched value, c.text and c.offset are NOT affected.

The default is false.

func Debug

func Debug(b bool) Option

Debug creates an Option to set the debug flag to b. When set to true, debugging information is printed to stdout while parsing.

The default is false.

func Entrypoint

func Entrypoint(ruleName string) Option

Entrypoint creates an Option to set the rule name to use as entrypoint. The rule name must have been specified in the -alternate-entrypoints if generating the parser with the -optimize-grammar flag, otherwise it may have been optimized out. Passing an empty string sets the entrypoint to the first rule in the grammar.

The default is to start parsing at the first rule in the grammar.

func GlobalStore

func GlobalStore(key string, value interface{}) Option

GlobalStore creates an Option to set a key to a certain value in the globalStore.

func InitState

func InitState(key string, value interface{}) Option

InitState creates an Option to set a key to a certain value in the global "state" store.

func MaxExpressions

func MaxExpressions(maxExprCnt uint64) Option

MaxExpressions creates an Option to stop parsing after the provided number of expressions have been parsed, if the value is 0 then the parser will parse for as many steps as needed (possibly an infinite number).

The default for maxExprCnt is 0.

func Memoize

func Memoize(b bool) Option

Memoize creates an Option to set the memoize flag to b. When set to true, the parser will cache all results so each expression is evaluated only once. This guarantees linear parsing time even for pathological cases, at the expense of more memory and slower times for typical cases.

The default is false.

func Recover

func Recover(b bool) Option

Recover creates an Option to set the recover flag to b. When set to true, this causes the parser to recover from panics and convert it to an error. Setting it to false can be useful while debugging to access the full stack trace.

The default is true.

func Statistics

func Statistics(stats *Stats, choiceNoMatch string) Option

Statistics adds a user provided Stats struct to the parser to allow the user to process the results after the parsing has finished. Also the key for the "no match" counter is set.

Example usage:

input := "input"
stats := Stats{}
_, err := Parse("input-file", []byte(input), Statistics(&stats, "no match"))
if err != nil {
    log.Panicln(err)
}
b, err := json.MarshalIndent(stats.ChoiceAltCnt, "", "  ")
if err != nil {
    log.Panicln(err)
}
fmt.Println(string(b))

type Protocol

type Protocol int
const (
	Unspecified Protocol = 0
	TCP         Protocol = iota
	UDP         Protocol = iota
	TLS         Protocol = iota
	WebSocket   Protocol = iota
)

func (Protocol) String

func (p Protocol) String() string

type Stats

type Stats struct {
	// ExprCnt counts the number of expressions processed during parsing
	// This value is compared to the maximum number of expressions allowed
	// (set by the MaxExpressions option).
	ExprCnt uint64

	// ChoiceAltCnt is used to count for each ordered choice expression,
	// which alternative is used how may times.
	// These numbers allow to optimize the order of the ordered choice expression
	// to increase the performance of the parser
	//
	// The outer key of ChoiceAltCnt is composed of the name of the rule as well
	// as the line and the column of the ordered choice.
	// The inner key of ChoiceAltCnt is the number (one-based) of the matching alternative.
	// For each alternative the number of matches are counted. If an ordered choice does not
	// match, a special counter is incremented. The name of this counter is set with
	// the parser option Statistics.
	// For an alternative to be included in ChoiceAltCnt, it has to match at least once.
	ChoiceAltCnt map[string]map[string]int
}

Stats stores some statistics, gathered during parsing

type StreamParseError

type StreamParseError struct{}

func (StreamParseError) Error

func (e StreamParseError) Error() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址