Documentation
¶
Overview ¶
Package clienterrorbus provides business access to client error monitoring — the browser application's own crashes, grouped into issues.
This is not customer log data. It is our own application reporting on itself, which is why it lives apart from logbus: the volume is different, the retention is shorter, the privacy rules are stricter (no PII at all), and an event is worth nothing on its own — the value is the issue it groups into.
Index ¶
- Constants
- Variables
- func Fingerprint(ne NewEvent) (fingerprint, title, culprit string)
- func NormalizeMessage(msg string) string
- func ParseStatus(s string) (string, error)
- func ResolveStack(ctx context.Context, stack, release string, maps SourceMapLookup) (out string, resolved bool)
- func ScrubText(s string) string
- func ScrubURL(s string) string
- func SpikeBounds(cfg SpikeConfig, now time.Time) (baselineStart, windowStart, windowEnd time.Time)
- func ValidKind(s string) bool
- func ValidLevel(s string) bool
- type APIContext
- type Artifact
- type Breadcrumb
- type Bucket
- type Business
- func (b *Business) EvaluateSpikes(ctx context.Context, cfg SpikeConfig) (int, error)
- func (b *Business) Ingest(ctx context.Context, who Reporter, events []NewEvent) (int, error)
- func (b *Business) ProcessBatch(ctx context.Context, limit int) (int, error)
- func (b *Business) PurgeOrg(ctx context.Context, orgID uuid.UUID) (int64, error)
- func (b *Business) QueryArtifacts(ctx context.Context, release string) ([]Artifact, error)
- func (b *Business) QueryIssueByID(ctx context.Context, id uuid.UUID) (Issue, error)
- func (b *Business) QueryIssueEvents(ctx context.Context, issueID uuid.UUID, limit int) ([]Event, error)
- func (b *Business) QueryIssueSeries(ctx context.Context, issueID uuid.UUID, from, to time.Time, ...) ([]Bucket, error)
- func (b *Business) QueryIssues(ctx context.Context, f IssueFilter) ([]Issue, string, error)
- func (b *Business) QuerySpikes(ctx context.Context, cfg SpikeConfig) ([]Spike, error)
- func (b *Business) QueryStats(ctx context.Context, orgID, projectID *uuid.UUID, allOrgs bool, ...) (Stats, error)
- func (b *Business) UpdateIssue(ctx context.Context, i Issue, ui UpdateIssue) (Issue, error)
- func (b *Business) UploadArtifact(ctx context.Context, na NewArtifact) (Artifact, error)
- type Event
- type Frame
- type Issue
- type IssueFilter
- type NewArtifact
- type NewEvent
- type Notifier
- type Reporter
- type SourceMapLookup
- type Spike
- type SpikeConfig
- type Stats
- type Storer
- type UpdateIssue
Constants ¶
const ( // MaxBatchEvents caps events per request. MaxBatchEvents = 50 // MaxBodyBytes caps the request body. MaxBodyBytes = 100 * 1024 // MaxMessageLen and MaxStackLen bound the two fields that are large by // nature. A stack deeper than this is not more diagnostic. MaxMessageLen = 1000 MaxStackLen = 8000 // MaxComponentStackLen bounds the React component stack. MaxComponentStackLen = 4000 // MaxBreadcrumbs and MaxBreadcrumbLen bound the trail of recent actions. MaxBreadcrumbs = 30 MaxBreadcrumbLen = 300 // MaxURLLen and MaxUserAgentLen bound the context fields. MaxURLLen = 500 MaxUserAgentLen = 400 // MaxReleaseLen bounds the release identifier, which is also a grouping key. MaxReleaseLen = 120 )
Limits on what a single report may carry. They are deliberately small: this endpoint is unauthenticated, so every field is a place for someone to put a megabyte.
const ( LevelFatal = "fatal" LevelError = "error" LevelWarning = "warning" )
Levels an event may report.
const ( KindUnhandled = "unhandled" KindUnhandledRejection = "unhandledrejection" KindReact = "react" KindAPI = "api" KindManual = "manual" )
Kinds of error the client distinguishes.
const ( StatusUnresolved = "unresolved" StatusResolved = "resolved" StatusIgnored = "ignored" )
Issue statuses.
const ( SortLastSeen = "lastSeen" SortCount = "count" SortUsers = "users" )
Sort orders for an issue listing.
const ( DefaultSpikeWindow = 10 * time.Minute DefaultSpikeBaseline = time.Hour DefaultSpikeMultiplier = 5.0 DefaultSpikeMinEvents = 25 )
Defaults for spike detection. Ten minutes is short enough to catch a bad deploy while it is still rolling out and long enough that a single unlucky minute does not read as a trend.
const FingerprintVersion = 1
FingerprintVersion is stamped on every event so a later pass can tell which events were grouped by which algorithm.
It will be bumped when source maps land: today's frames are minified, so a fingerprint built from them changes at every deploy — the same bug in index-64s.js and index-9f2.js looks like two issues. Version 1 accepts that, because grouping by minified frames is still far better than grouping by nothing, and the re-group is a background job over stored events rather than data we have to collect again.
const MaxArtifactBytes = 32 * 1024 * 1024
MaxArtifactBytes caps one uploaded map. Bundles get large, but a map an order of magnitude past this is a mistake rather than a build.
const Redacted = "[redacted]"
Redacted is what a scrubbed secret is replaced with.
const ResolvedFingerprintVersion = 2
ResolvedFingerprintVersion is stamped on events grouped from de-minified frames. Events at a lower version were grouped from minified ones and are re-grouped when a map for their release arrives.
Variables ¶
var ( ErrArtifactTooLarge = fmt.Errorf("a source map may be at most %d bytes", MaxArtifactBytes) ErrArtifactEmpty = errors.New("source map is empty") ErrArtifactRelease = errors.New("release is required") ErrArtifactInvalid = errors.New("file is not a valid source map") )
Errors returned when uploading a source map.
var ( ErrNotFound = errors.New("issue not found") ErrNoEvents = errors.New("batch contained no events") ErrTooManyEvents = fmt.Errorf("a batch may carry at most %d events", MaxBatchEvents) ErrInvalidStatus = errors.New("status must be unresolved, resolved, or ignored") )
Set of error variables for CRUD operations.
Functions ¶
func Fingerprint ¶
Fingerprint returns the grouping key for an event, plus the title and culprit derived from the same material.
The inputs are the error type, the kind of failure, the normalized message and the top in-app frames. Kind is included because an api failure and a render crash with the same message are different problems with different owners.
func NormalizeMessage ¶
NormalizeMessage reduces a message to a grouping template.
func ParseStatus ¶
ParseStatus validates a submitted issue status.
func ResolveStack ¶
func ResolveStack(ctx context.Context, stack, release string, maps SourceMapLookup) (out string, resolved bool)
ResolveStack rewrites a minified stack using the maps for its release.
Frames that cannot be resolved are kept exactly as they were: a partially resolved stack is far more useful than none, and it is normal for one frame to come from a file we have no map for. resolved reports whether anything at all was rewritten, which is what decides the fingerprint version.
One case does not resolve and cannot be made to: a column past the last mapping on its generated line. The lookup finds the nearest mapping at or before a column, and there is nothing before the end of the table to fall back to. In a real bundle line 1 carries thousands of mappings, so this only affects a frame in the final expression of the file.
func ScrubURL ¶
ScrubURL reduces a reported location to a path. The client is asked to send a path only; this enforces it, because a URL is where session tokens end up in practice.
func SpikeBounds ¶
func SpikeBounds(cfg SpikeConfig, now time.Time) (baselineStart, windowStart, windowEnd time.Time)
SpikeBounds exposes the window arithmetic to the store, which needs the same three instants, and to tests, which need to place events inside them.
func ValidLevel ¶
ValidLevel reports whether a submitted level is one we accept.
Types ¶
type APIContext ¶
type APIContext struct {
Method string `json:"method,omitempty"`
Path string `json:"path,omitempty"`
Status int `json:"status,omitempty"`
Code string `json:"code,omitempty"`
}
APIContext describes the failed request behind an api-kind error.
type Artifact ¶
type Artifact struct {
ID uuid.UUID
Release string
FileName string
ByteSize int
UploadedBy string
DateCreated time.Time
}
Artifact is one uploaded source map.
type Breadcrumb ¶
Breadcrumb is one recent action leading up to an error.
type Business ¶
type Business struct {
// contains filtered or unexported fields
}
Business manages the set of APIs for client error monitoring.
func NewBusiness ¶
NewBusiness constructs a client error business API. notifier may be nil, in which case grouping still happens and nothing is delivered.
func (*Business) EvaluateSpikes ¶
EvaluateSpikes finds issues whose rate has jumped and notifies each once.
It runs on a timer for the same reason threshold rules do: a rate is a property of a period, not of the event that happens to be the hundredth. An issue that stays elevated is not re-notified on every pass — the dedup window in the delivery path governs that, exactly as it does for every other alert.
func (*Business) Ingest ¶
Ingest validates, scrubs and stores a batch, returning how many events were accepted. Duplicates by event id are not counted twice.
Nothing here computes a fingerprint or writes an issue. The browser is waiting on this call, and a poisoned event must not be able to fail the request that delivered it — grouping happens on the worker.
func (*Business) ProcessBatch ¶
ProcessBatch groups up to limit unprocessed events into issues and returns how many it handled. It is safe to run concurrently: the claim skips rows another worker holds.
func (*Business) PurgeOrg ¶
PurgeOrg deletes every client error record belonging to an org, for a deletion request. Issues cascade from the org, taking their facets with them.
func (*Business) QueryArtifacts ¶
QueryArtifacts lists what has been uploaded for a release, so a deploy can be checked without a crash to test it on.
func (*Business) QueryIssueByID ¶
QueryIssueByID returns one issue.
func (*Business) QueryIssueEvents ¶
func (b *Business) QueryIssueEvents(ctx context.Context, issueID uuid.UUID, limit int) ([]Event, error)
QueryIssueEvents returns an issue's most recent events.
func (*Business) QueryIssueSeries ¶
func (b *Business) QueryIssueSeries(ctx context.Context, issueID uuid.UUID, from, to time.Time, interval time.Duration) ([]Bucket, error)
QueryIssueSeries returns an issue's count over time.
func (*Business) QueryIssues ¶
QueryIssues lists issues matching the filter.
func (*Business) QuerySpikes ¶
QuerySpikes returns the issues whose rate has jumped, without notifying. It is what the evaluator reads, and what a dashboard would read to badge an issue as spiking rather than waiting for an alert to arrive.
func (*Business) QueryStats ¶
func (b *Business) QueryStats(ctx context.Context, orgID, projectID *uuid.UUID, allOrgs bool, from, to time.Time) (Stats, error)
QueryStats returns the dashboard tiles for a window.
func (*Business) UpdateIssue ¶
UpdateIssue applies a triage decision.
Resolving clears the regressed flag: the point of the flag is to say "this came back after you closed it", and closing it again is the acknowledgement.
func (*Business) UploadArtifact ¶
UploadArtifact stores one source map for a release and queues that release's existing events to be grouped again.
Two things happen to the file on the way in. Its embedded sourcesContent is dropped, and it is gzipped. Dropping the sources is the important one: it is the bulk of the file and we never read it, so keeping it would mean holding a copy of the frontend source at rest for no purpose. What remains is the mapping table, which is what resolution actually needs.
type Event ¶
type Event struct {
ID uuid.UUID
EventID uuid.UUID
OrgID *uuid.UUID
ProjectID *uuid.UUID
UserID *uuid.UUID
Role string
Level string
Kind string
Name string
Message string
Stack string
ComponentStack string
// ResolvedStack is the de-minified stack, empty when there was no source map
// for the event's release. The raw Stack is always kept.
ResolvedStack string
Release string
Environment string
URL string
UserAgent string
API *APIContext
Breadcrumbs []Breadcrumb
OccurredAt time.Time
ReceivedAt time.Time
Fingerprint string
IssueID *uuid.UUID
SampledCount int
}
Event is a stored error report.
type Frame ¶
type Frame struct {
Function string
File string
Line string
Column string
// InApp is false for frames belonging to a dependency.
InApp bool
}
Frame is one parsed stack frame.
func InAppFrames ¶
InAppFrames returns the frames belonging to our own code, falling back to every frame when none qualify — a stack made entirely of vendor frames still has to group as something, and dropping it would lose the event.
func ParseStack ¶
ParseStack extracts frames from a browser stack trace, oldest last, in the order the browser reported them.
type Issue ¶
type Issue struct {
ID uuid.UUID
OrgID *uuid.UUID
ProjectID *uuid.UUID
Fingerprint string
Title string
Culprit string
Level string
Kind string
Status string
Regressed bool
EventCount int64
FirstSeenAt time.Time
LastSeenAt time.Time
ResolvedAt *time.Time
AssigneeID *uuid.UUID
SampleEventID *uuid.UUID
// Counts of distinct facets, filled by the read path.
AffectedUsers int
AffectedOrgs int
Releases []string
}
Issue is a group of events sharing one fingerprint.
type IssueFilter ¶
type IssueFilter struct {
// OrgID scopes the listing. Nil means the anonymous bucket, which only a
// super admin may read.
OrgID *uuid.UUID
// ProjectID narrows to one project within the org.
ProjectID *uuid.UUID
Status string
// Release, when set, restricts to issues seen on that release.
Release string
// Since bounds by last seen.
Since *time.Time
// Sort is one of "lastSeen" (default), "count" or "users".
Sort string
Limit int
// Cursor is the opaque page cursor from a previous response.
Cursor string
// AllOrgs lets a super admin read across every org, including anonymous.
AllOrgs bool
}
IssueFilter narrows an issue listing.
type NewArtifact ¶
type NewArtifact struct {
Release string
// FileName is the generated file the map describes, as it appears in a stack
// trace. A ".map" suffix is stripped, because that is what CI has on disk and
// "index-64s.js" is what the browser reports.
FileName string
Content []byte
UploadedBy string
}
NewArtifact is one source map being uploaded.
type NewEvent ¶
type NewEvent struct {
EventID uuid.UUID
OccurredAt time.Time
Level string
Kind string
Name string
Message string
Stack string
ComponentStack string
Release string
Environment string
URL string
UserAgent string
API *APIContext
Breadcrumbs []Breadcrumb
SampledCount int
}
NewEvent is one submitted error report, before it is scrubbed and stored.
UserID, OrgID and Role arrive as client hints and are overwritten by the server from the presented token; the client's copy is never trusted.
type Notifier ¶
type Notifier interface {
IssueOpened(ctx context.Context, i Issue, sample Event)
IssueRegressed(ctx context.Context, i Issue, sample Event)
// IssueSpiked reports a known issue whose rate has jumped. It is the third
// trigger and the one that catches "the deploy made an existing bug much
// worse" — which is neither new nor a regression, so nothing else fires.
IssueSpiked(ctx context.Context, i Issue, spike Spike)
}
Notifier is told about issues worth waking someone for. It is an interface so this package does not depend on the alerting domain: delivery, suppression and maintenance windows already exist and are not rebuilt here.
type Reporter ¶
type Reporter struct {
UserID *uuid.UUID
OrgID *uuid.UUID
// ProjectID is the project the user was working in. It is what makes a crash
// alertable: rules and delivery channels are project-scoped, so a report
// without one can be grouped and triaged but cannot page anybody.
ProjectID *uuid.UUID
Role string
}
Reporter is who the server decided the report came from, resolved from the token rather than from the payload.
type SourceMapLookup ¶
type SourceMapLookup interface {
MapFor(ctx context.Context, release, fileName string) (*sourcemap.Consumer, bool)
}
SourceMapLookup returns the map for one generated file of a release, already decompressed, or ok=false when there is none.
It is an interface so resolution can be tested without a database, and so the worker can cache: parsing a multi-megabyte map for every frame of every event would cost more than the grouping it feeds.
type Spike ¶
type Spike struct {
Issue Issue
// Current is how many events landed in the window.
Current int64
// Baseline is the per-window rate before it, scaled to the window's length so
// the two numbers are comparable.
Baseline float64
// Window is the period Current covers.
Window time.Duration
}
Spike is an issue whose rate has jumped.
type SpikeConfig ¶
type SpikeConfig struct {
// Window is the period the current rate is measured over.
Window time.Duration
// Baseline is how far back the comparison rate is drawn from, ending where
// the window begins.
Baseline time.Duration
// Multiplier is how many times the baseline rate counts as a spike.
Multiplier float64
// MinEvents is the floor: fewer than this in the window is never a spike,
// whatever the ratio.
MinEvents int
}
SpikeConfig is what counts as a spike.
The shape is relative with a floor, which is what people mean by "spiking": an absolute threshold has to be set per issue to be useful, and nobody sets it. The floor is what keeps a quiet issue going from one event an hour to ten — a tenfold rise, and nothing worth waking for — out of the alerts.
type Stats ¶
type Stats struct {
From time.Time
To time.Time
Events int64
Issues int64
NewIssues int64
Unresolved int64
AffectedUsers int
}
Stats are the dashboard tiles for a window.
type Storer ¶
type Storer interface {
Ingest(ctx context.Context, events []Event) (int, error)
ClaimUnprocessed(ctx context.Context, limit int, maxAttempts int) ([]Event, error)
AttachToIssue(ctx context.Context, eventID uuid.UUID, issueID uuid.UUID, fingerprint string, version int, resolvedStack string) error
DetachFromIssue(ctx context.Context, issueID uuid.UUID, count int) error
MarkFailed(ctx context.Context, eventID uuid.UUID, reason string, maxAttempts int) error
UpsertIssue(ctx context.Context, i Issue) (Issue, bool, error)
RecordFacets(ctx context.Context, issueID uuid.UUID, facets map[string][]string) error
TouchIssue(ctx context.Context, issueID uuid.UUID, at time.Time, level string, sampleEventID uuid.UUID, count int) (reopened bool, err error)
QueryIssueByID(ctx context.Context, id uuid.UUID) (Issue, error)
QueryIssues(ctx context.Context, f IssueFilter) ([]Issue, string, error)
UpdateIssue(ctx context.Context, i Issue) error
QueryIssueEvents(ctx context.Context, issueID uuid.UUID, limit int) ([]Event, error)
QueryIssueSeries(ctx context.Context, issueID uuid.UUID, from, to time.Time, interval time.Duration) ([]Bucket, error)
QueryStats(ctx context.Context, orgID, projectID *uuid.UUID, allOrgs bool, from, to time.Time) (Stats, error)
PurgeOrg(ctx context.Context, orgID uuid.UUID) (int64, error)
// Source maps and the re-grouping they trigger.
UpsertArtifact(ctx context.Context, a Artifact, content []byte) error
QueryArtifact(ctx context.Context, release, fileName string) ([]byte, error)
QueryArtifactsByRelease(ctx context.Context, release string) ([]Artifact, error)
MarkReleaseForRegroup(ctx context.Context, release string, belowVersion int) (int64, error)
QuerySpikes(ctx context.Context, cfg SpikeConfig, now time.Time) ([]Spike, error)
}
Storer declares the persistence behavior this package needs.
The event table doubles as the work queue: Ingest writes, ClaimUnprocessed takes a batch for grouping. There is no broker because there is nothing a broker would add here that a table with SKIP LOCKED does not already do, and an in-memory queue would make the 202 a lie the first time the process restarts.
type UpdateIssue ¶
type UpdateIssue struct {
Status *string
// AssigneeID is a double pointer so unassigning is distinguishable from
// leaving the assignee alone.
AssigneeID **uuid.UUID
}
UpdateIssue carries the fields a triaging human may change.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
stores
|
|
|
clienterrordb
Package clienterrordb contains client error monitoring database access.
|
Package clienterrordb contains client error monitoring database access. |