Repository navigation
feat(v2)!: sync v2 with idin-core — fixes, memory MQ, presigned POST, rate limit, retry - #17
Merged
Merged
Conversation
…r's label A repository failure is wrapped as "repository", so every database error reached Sentry titled "DATABASE_ERROR: repository" — the driver's message was only in the innermost exception. retitle now appends the root cause's value when the root is not a framework error, read from the already-scrubbed event. The client's message is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…y" label
mapErr wrapped every database failure with core.Wrap(err, "repository"),
which puts the label into Message — the field a response carries — so a
client read {"code":"DATABASE_ERROR","message":"repository"}.
core.WrapAs(err, sentinel, op) reports err as the sentinel (status, code,
message) and keeps op for Error(), Op(), the log and the Sentry title only.
mapErr now answers with errmsgs.DBError's message; the Sentry issue is still
titled "DATABASE_ERROR: repository: <driver error>". Wrap is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Trivy flagged nine advisories on v2/go.mod: ssh DoS in x/crypto, HTTP/2 DoS and flow-control bypasses in x/net, and a malformed-RPC DoS in grpc. x/crypto v0.58.0, x/net v0.61.0 and grpc v1.84.0 fix all of them. GO-2026-5932 (x/crypto/openpgp) has no fixed version and is ignored in v2/.trivyignore: Trivy matches the whole module, but no v2 package imports openpgp, so the code is never linked. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…r edge cases Bugs found by a full review of v2, each with a unit test: - repository.Transaction hands the callback a clean *gorm.DB (no Model/Where carried over) and returns an IError from the callback unchanged instead of wrapping it as DATABASE_ERROR - core.Wrap/From keep the status, code and fields of an IError of another type (valid.Errors) instead of downgrading it to 500 - Sentry: a shared sentinel (errmsgs.DBError) is never marked as captured; scrub keys match hyphenated names (X-Api-Key); AI, Slack and DSN secrets are masked by value; job params are scrubbed field by field - Runner.Stop ends RunContext for a process with no HTTP server and waits for the shutdown to finish - S3 Delete reports per-key failures from DeleteObjects; PublicURL escapes the key; the memory storage's ETag is the MD5 like S3's - Mongo UpdateOne/UpdateMany/ReplaceOne pass ArrayFilters, Hint and Collation to the driver; Find/Aggregate release their deadline when drained; the command monitor no longer stops logging forever when full - mongorepo: Updates/Inc/Push/Pull/AddToSet/Unset/DeleteOne refuse an empty scope (UNSCOPED_UPDATE / UNSCOPED_DELETE); Pipeline.One wraps ErrDocumentNotFound - jobstore: Update no longer resurrects a purged run; List checks order_by against the run columns and leaves the caller's options alone; the "trigger" column is quoted; the sqlite tests run under `make test` - context: SetUser/GetUser are synchronised, WithContext carries the data bag, NewApp(nil) uses an empty environment - memory pub/sub cannot send on a closed channel; valid.SetMessage is guarded by a mutex; the scheduler guards its job map - Auth middleware clones the verifier's user before stamping the token; BindOnly also binds `header:` fields; truncation never splits a rune; CSV float32 prints at its own precision; the postgres DSN quotes values; the tap handler honours WithGroup; devtools hides *_pass/*_key and URLs carrying credentials; basic auth keeps the code's user when the password comes from the environment; postmangen recognises BindOnly - coretest writes its env to a file instead of t.Setenv, so fixtures work under t.Parallel() BREAKING CHANGE: visible behaviour that consumers may have relied on: - A SQL unique-constraint violation is 409 DUPLICATE_KEY (errmsgs.DuplicateKey) instead of 500 DATABASE_ERROR; FK violations are 409 FOREIGN_KEY_VIOLATED and check constraints 400 CHECK_CONSTRAINT_VIOLATED (gorm TranslateError is on). Migrate: branch on the new codes, or keep a pre-insert existence check. - Requester.Send maps a cancelled caller context to 499 REQUEST_CANCELED and an exceeded deadline to 504 UPSTREAM_TIMEOUT instead of 500 NETWORK_ERROR. - GetPageOptionsWithAllowed() with no columns allows no client ordering; use GetPageOptions() for "any column". - mongorepo single-document writes without a filter return 400 instead of changing an arbitrary document; use UpdateAll/DeleteAll/Upsert/Save on purpose. - repository.Transaction's tx no longer carries the repository's Model and Where; build repositories on it with NewWithDB as documented. - coretest.WithEnv values no longer override APP_* variables already set in the process; the process environment wins, as for a real service. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ers, retry, probe log skipping - core.NewMemoryMQ(): an in-process broker (direct/fanout/topic/headers routing, bindings, dead-lettering, TTL, MaxLength, Mandatory). App.NewMQConsumer consumes from it when the App's publisher is a MemoryMQ, so a service's OnQueue/Start/Requeue wiring is tested without RabbitMQ. Messages(queue), Published() and Reset() are the assertion helpers. The RabbitMQ consumer's dispatch is shared with it (runMQHandler / logMQOutcome). - IStorage.PresignPost: a browser upload form with a policy the bucket enforces (content type or prefix, size range, key prefix, metadata, redirect). Implemented for S3, the memory storage and the disabled one. - core.RateLimit(cache, limit, window, opts...): fixed-window limiter on ICache.Incr with X-RateLimit-* and Retry-After headers, per route per key, fail-open. core.SecureHeaders(): X-Frame-Options, nosniff, Referrer-Policy, HSTS on TLS only, optional CSP/Permissions-Policy. - core.Retry(ctx, RetryPolicy, fn): exponential backoff with full jitter, RetryStop, RetryIf, OnRetry; *Error.WithRetryAfter / RetryAfterOf carry a 429's wait. Requester retries transport failures, 429 and 5xx for idempotent requests via HTTP_RETRY_COUNT/HTTP_RETRY_WAIT_MS/ HTTP_RETRY_MAX_WAIT_MS, honouring Retry-After. Mailer retries transient send failures via EMAIL_RETRY_COUNT / WithMailRetries. The Slack driver waits for a short Retry-After and tries again (CHAT_SLACK_RETRIES, CHAT_SLACK_RETRY_MAX_WAIT). - LOG_REQUEST_SKIP_PATHS / HTTPOptions.SkipRequestLogPaths drop the access line for successful requests on the listed paths (/healthz, /readyz, /_dev/*); a failing probe is still logged. BREAKING CHANGE: IStorage gained the PresignPost method. A type outside this module that implements IStorage itself must add it (delegate to noopStorage-style StorageDisabled, or implement it); the framework's own NewStorage/NewMemoryStorage callers are unaffected. The Slack driver now retries a 429 by default (2 attempts, waits up to 10s); set CHAT_SLACK_RETRIES=0 to keep the previous behaviour. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Ported from idin-core 5a1540f. - toolchain go1.27.2 fixes the 12 stdlib vulnerabilities govulncheck reported against 1.27.1 (net/http, http2, crypto/tls, html/template) - direct dependencies updated: aws-sdk-go-v2, koanf, goai 0.10.8, google api 0.304, plus x/net 0.61 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
govulncheck reports GO-2026-6809 (crypto/x509 trusts compromised
intermediates) against go1.27.2, reachable through the HTTP server.
go1.27.3 fixes it.
golangci-lint v2.13.2 cannot read the export data that the go1.27.2+
standard library ships ("export data version 5 is greater than maximum
supported version 4"), so every package failed typecheck. v2.15.0 reads it
and reports no issues on v2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
v2 was imported from idin-core at
daf4840. Since then, idin-core has fixed bugs found in a full review of v2, added several capabilities and patched published CVEs. This PR ports those six v2 commits with the module path rewritten togithub.com/pskclub/mine-core/v2:DBErrorinstead of the label "repository"Runner.Stop, S3 delete/URL, Mongo update options, mongorepo unscoped writes, jobstore, context races, Sentry scrubbing, and more. Each fix has a unit test.core.NewMemoryMQ(),IStorage.PresignPost,core.RateLimit,core.SecureHeaders,core.Retry, plus retries in Requester, Mailer and Slack. Probe requests can be left out of the access log withLOG_REQUEST_SKIP_PATHS.v2/.trivyignoreignores GO-2026-5932 (openpgp, which v2 never links).v2/sentry.gokeeps mine-core's existing comment on the sentry-go logs/metrics opt-out. Only the comment differs; the code is the same.BREAKING CHANGE: visible behaviour that consumers may have relied on:
IStoragegainedPresignPost. A type outside this module that implementsIStorageitself must add this method. Callers ofNewStorage/NewMemoryStorageare unaffected.DUPLICATE_KEYinstead of 500DATABASE_ERROR. A foreign-key violation is 409FOREIGN_KEY_VIOLATED, and a check-constraint violation is 400CHECK_CONSTRAINT_VIOLATED. To migrate, branch on the new codes.Requester.Sendreturns 499REQUEST_CANCELEDwhen the caller cancels and 504UPSTREAM_TIMEOUTwhen the deadline passes, instead of 500NETWORK_ERROR.GetPageOptionsWithAllowed()with no columns allows no client ordering. UseGetPageOptions()to allow ordering on any column.UpdateAll/DeleteAll/Upsert/Savewhen you mean to write without one.txpassed byrepository.Transactionno longer carries the repository'sModel/Where. Build repositories on it withNewWithDB.coretest.WithEnvno longer overridesAPP_*variables that are already set in the process.CHAT_SLACK_RETRIES=0to turn this off.Module
Checklist
go build,go vet(with and without the integration/e2e tags),gofmtandgo test ./...pass locallyv2/docs/is updated for anything a user of the framework would notice!, and the description has aBREAKING CHANGE:section with migration steps🤖 Generated with Claude Code