Documentation
¶
Overview ¶
Package project exposes read-only Project membership facts that other modules need without importing modules/project.
This package must never import modules/project (pinned by TestPkgProjectDoesNotImportModulesProject); doing so would put the import cycle back.
Index ¶
- Constants
- Variables
- func EpochsReadTimeout() time.Duration
- func FoldID(id string) string
- func FoldedHas(set map[string]bool, want string) bool
- func FoldedLookup(set map[string]bool, want string) (string, bool)
- func MembershipsInSpace(session *dbr.Session, spaceID, uid string, projectIDs []string) (map[string]Membership, error)
- func ProjectEpochsInSpace(session dbr.SessionRunner, spaceID string, projectIDs []string) (map[string]int64, error)
- func ProjectMemberships(ctx context.Context, session *dbr.Session, spaceID, projectID string, ...) (int64, map[string]int, error)
- func RepairAbsentSentinelEpoch(ctx context.Context, session *dbr.Session, projectID string) (bool, error)
- func SetMembershipTearHookForTest(fn func()) (restore func(), err error)
- type Membership
- type ProjectMetadata
- type SentinelAnomalyError
- type UserProject
Constants ¶
const AbsentEpochSentinel = 0
AbsentEpochSentinel is the value the membership integration contract reserves for "this project does not exist or is not visible".
It is a value in the same domain as a real epoch, which is what makes ErrLiveProjectOnAbsentSentinel necessary.
Variables ¶
var ErrLiveProjectOnAbsentSentinel = errors.New(
"project: an active project holds the reserved absent-epoch sentinel")
ErrLiveProjectOnAbsentSentinel is returned when an ACTIVE project is found holding the reserved value, i.e. when the answer these functions would give is indistinguishable from "gone" for a project that is not.
Callers MUST fail the request rather than serve the value. See ProjectEpochsInSpace for why the row can exist at all and why the read layer is where it has to be refused.
Functions ¶
func EpochsReadTimeout ¶
EpochsReadTimeout is the same cutoff for the epochs endpoint's transaction.
Exported, and deliberately the SAME constant rather than a second literal: modules/internal_membership opens that transaction itself (it owns the store), and two independently drifting deadlines on the two halves of one peer contract is how the pair stops meaning anything. A named accessor rather than an exported var so it cannot be reassigned at runtime.
func FoldID ¶
FoldID normalizes an identifier for MATCHING, and it is the only fold in this package — the read functions below key their answers by it so no caller has to re-derive the rule.
Why a fold is needed at all ¶
octo_project and octo_project_member are pinned to utf8mb4_general_ci, which is case-INSENSITIVE. `project_id IN (?)` therefore matches a stored `abc` when the caller sends `ABC`, and the row comes back spelled `abc`. Keying an answer off the spelling the DATABASE returned means a caller looking up its own `ABC` finds nothing — and an absent key is the "does not exist" / member:false sentinel, so a real member of a live project would read as denied while the SQL had matched perfectly.
ASCII-only, and this time actually ¶
This was strings.ToLower, described in a comment as "ASCII-only folding" and as "the STRICTER of the two, so the disagreement costs an answer of absent rather than a wrong positive". All three claims were false: strings.ToLower is UNICODE case mapping, and it is LOOSER than utf8mb4_general_ci. Measured on MySQL 8.0.33:
Go: strings.ToLower("\u212A") == "k" -> true (KELVIN SIGN)
Go: strings.ToLower("\u212B") == "å" -> true (ANGSTROM SIGN)
MySQL: (_utf8mb4 0xE284AA) COLLATE utf8mb4_general_ci = 'k' -> 0
MySQL: (_utf8mb4 0xE284AB) COLLATE utf8mb4_general_ci = 'å' -> 0
So the fold merged identifiers the database keeps apart, and on _verify that is fail-OPEN: send uids ["k", "\u212A"], SQL matches only the real row `k`, both requested ids fold to "k", and the id the database never matched is served as a member WITH a role.
A byte-level ASCII fold instead. Non-ASCII bytes are left untouched, which makes this strictly COARSER-than-nothing and strictly FINER than either collation (general_ci also folds accents; 0900_ai_ci folds accents and the compatibility characters above). Finer means a disagreement can only cost an "absent" answer, never a wrong positive — the direction the comment always claimed.
Two spellings cannot collide into one wrong answer either: project_id and (project_id, uid) are UNIQUE under general_ci, whose equivalence classes are a superset of this fold's, so the database cannot hold two rows that this folds together.
Rejecting non-canonical case at the boundary was the alternative and it is worse: it would break a caller holding a legitimately re-cased id, and nothing here has the standing to declare one spelling canonical — existing ids predate the UUID format and nothing validates their shape.
func FoldedHas ¶
FoldedHas reports whether want is present in a set keyed by the spelling the DATABASE returned.
Every caller that intersects two of those sets needs this, and open-coding it is how one of the sites ends up exact-match: `space_member`, `user` and `octo_project_member` all compare `uid` case-insensitively, so a row can come back spelled differently from the uid that was asked for, and an exact lookup then drops a live, seated member. The direction is fail-closed — a real caller refused — which is the shape this package already treats as a defect on the read path.
Folding the SET rather than the needle, because the set is the side with the database's spelling and there may be many of them.
func FoldedLookup ¶
FoldedLookup is FoldedHas that also returns the DATABASE's spelling.
Callers that go on to WRITE the uid into another table need this rather than the boolean: the two tables do not share a collation (octo_* are pinned utf8mb4_general_ci, the dump-imported legacy tables are utf8mb4_0900_ai_ci in production), so a row written with the caller's spelling can be unreachable from a query that resolved the same person through the other table. Storing what the database returned keeps the two byte sequences identical instead of relying on a collation to bridge them.
The miss direction is unchanged and still fail-closed: FoldID is ASCII-only, so a spelling that only the looser collation considers equal does not match here and the caller refuses. That refusal is what keeps non-canonical spellings out of the octo_* tables today; returning the canonical one is what stops that from being the only thing keeping them out.
func MembershipsInSpace ¶
func MembershipsInSpace(session *dbr.Session, spaceID, uid string, projectIDs []string) (map[string]Membership, error)
MembershipsInSpace returns, for the named projects, the ones where uid holds an active seat — keyed by project_id. Absent from the map means "not a member", for any reason.
One query for the whole batch. This feeds /v1/auth/verify, which every subsystem that fronts octo-server calls on EVERY request, so a per-id loop would multiply the gateway's database load by the number of projects a request happens to mention.
The Space filter is part of the predicate rather than a separate check: a project in another Space must be indistinguishable from one that does not exist, and the cheapest way to guarantee that is for both to produce the same absence rather than two branches that could drift.
`removing = 0` is part of the membership fact: a seat being closed is not a member, and a consumer that disagreed would authorize access to a Project whose memberships are being torn down.
func ProjectEpochsInSpace ¶
func ProjectEpochsInSpace(session dbr.SessionRunner, spaceID string, projectIDs []string) (map[string]int64, error)
ProjectEpochsInSpace returns member_epoch for each named ACTIVE project in spaceID, keyed by FoldID(project_id) — NOT by the spelling the caller sent and NOT by the spelling the database returned. Callers look up with FoldID too; see FoldID for why neither raw spelling works.
Absent from the map means the project does not exist, is disbanded, or lives in another Space. The caller maps all three to epoch 0 — one indistinguishable answer, for the same reason MembershipsInSpace folds them together: telling them apart would let a caller probe which Space a project id lives in.
`status = 1` is what makes project disband converge without a separate event: a disbanded project drops out of this result, the caller reads 0, and an authorization snapshot taken against the old epoch stops matching.
The PARENT Space is part of the answer, and it is checked last ¶
A Space ban flips `space.status` and touches nothing else — no octo_project write, no epoch bump, and the member-removal cascade deliberately skips banned Spaces so an unban can restore them. A Space disband is worse: nothing disbands the projects of a disbanded Space, so their rows stay `status = 1` forever.
Without the parent check these two functions DISAGREE, and the disagreement lands on the channel the peer uses to invalidate:
ban: ProjectMemberships flips to member:false (its Space conjunction sees
the ban) while this function keeps answering the same epoch E. The
peer re-checks, E == E, its check AGREES, and the cached grant
survives the ban — unbounded.
unban: a peer that re-verified during the ban cached member:false under E.
The unban restores the answer, the epoch is still E, the check agrees
again, and every member of every project in that Space stays denied.
The unban direction was introduced by adding the Space conjunction to ProjectMemberships alone: before that both functions ignored Space status and at least agreed with each other. One predicate is not allowed to know something the invalidation channel does not.
Checked LAST, after the project rows, for the same reason ProjectMemberships reads its Space half last: this read can only ever REMOVE projects from the answer, so the freshest data arriving here is the fail-closed direction. A ban committing between the two reads yields "rows, then inactive" — everything drops, the peer reads 0, re-verifies. The other order would yield "active, then rows" and serve a live epoch for a Space that is already banned.
A single-row lookup on `space` rather than a JOIN, deliberately: every project in one call shares one space_id, so the join would buy nothing, and `octo_project` pins utf8mb4_general_ci while `space` is a 2019 table that inherits the server default — measured as utf8mb4_0900_ai_ci in production. An implicit cross-schema comparison is MySQL error 1267 THERE while passing in CI, which on a fail-closed endpoint means the peer is denied everything and the lane that would have caught it is green. See modules/project/reconcile_p1.go for the repository's full account of that trap.
An ACTIVE project on the sentinel is refused, not served ¶
That whole scheme rests on a live project never holding 0, and it is enforced by three writers in modules/project — creation bumps the epoch, a migration lifted the existing rows, and the reconcile scan repairs regressions. The first two are one-shots, so there is a window: a not-yet-upgraded pod mid rollout, or a rolled-back binary, inserts at the column default again, and the scan closes that only on its next rotation.
Serving such a row is NOT merely an availability problem, which an earlier version of this reasoning claimed. It is the stale-grant direction, reachable in that window and permanent once it happens:
- an old pod creates P at member_epoch 0, status 1;
- the peer verifies, is told epoch 0, and caches a positive grant at 0;
- P is disbanded before the scan reaches it — disband bumps the epoch and then flips status, so the row becomes epoch 1, status 0;
- the peer re-reads: the status filter drops P, and the answer is 0;
- 0 == 0, so the staleness check AGREES and the grant never expires.
The repair scan cannot fix this after the fact: its predicate needs status = 1, and by step 3 the row is disbanded forever.
So the sentinel is refused at the read layer. The window then costs availability instead of costing a permanent grant, which is the trade this whole module makes everywhere else.
How long that window actually is: the peer gets a 500 and retries, and the handler repairs the named row out of band on the way out (modules/internal_membership -> RepairAbsentSentinelEpoch), so the practical bound is ONE request. Do NOT read it as "the reconcile scan fixes it within one rotation" — that claim was wrong and is corrected on SentinelAnomalyError above: scanEpochSanity walks a bounded page budget per tick behind a persisted cursor, a row written by a not-yet-upgraded pod carries the highest id, and on a large octo_project reaching it takes hours. The refusal is also per-BATCH, so during that window every request whose batch of 50 contains the id fails, not just requests naming it.
It also removes the rollback runbook's dependency on the reconcile loop still being enabled: with the loop off, the endpoint refuses instead of silently handing out the collision.
func ProjectMemberships ¶
func ProjectMemberships(ctx context.Context, session *dbr.Session, spaceID, projectID string, uids []string) (int64, map[string]int, error)
ProjectMemberships answers, for ONE project, which of the named uids hold an active seat — the mirror image of MembershipsInSpace, which answers one uid across many projects.
Returns epoch 0 and an empty map when the project is not an active project of spaceID. The roles map is keyed by FoldID(uid), like ProjectEpochsInSpace keys its own answer and for the same reason — see FoldID.
Read order is load-bearing ¶
The epoch is read BEFORE the member rows, and that order must not be swapped. A membership write bumps member_epoch in the same transaction, so a consumer caches this answer under the epoch and re-checks the epoch later to decide whether the cache is still good. The invariant that makes that sound is:
the returned epoch is never NEWER than the returned membership data.
Read epoch first and a change landing between the two queries yields an OLD epoch beside NEW membership: the consumer's next epoch check differs, so it re-verifies. One wasted round trip, no stale grant.
Read members first and the same interleaving yields NEW epoch beside OLD membership: the consumer caches a stale positive under the current epoch, and its epoch check keeps agreeing. A removed member stays authorized until the next unrelated bump. That is the bug this ordering exists to prevent, and it is invisible in a single-threaded test — see the ordering guard test.
`removing = 0` is here for the same reason as every other predicate in this package: a seat being closed is not a member.
The Space half is conjoined here, and only here ¶
A project seat alone is NOT the answer to "may this uid act in this project". The Space→project cascade is ASYNCHRONOUS: when a user is removed from a Space, the project seat survives until a background job gets to it, that job's cleanup step deactivates the seat directly without the synchronous `removing = 1` phase (modules/project/space_member_removal.go), and it gives up after removalCleanupMaxAttempts with the row kept but never re-claimed. So `status = 1 AND removing = 0` on its own reports a removed user as a member for a window that is unbounded in the failure case.
Every OTHER caller of that predicate runs downstream of a Space gate — the project routes go through spacepkg.CheckMembership in middleware, and group admission conjoins spacepkg.ActiveMembers explicitly. This function is the first caller with no gate in front of it: its consumer is a peer control plane asking about a THIRD party, for whom it holds no token, and no endpoint in this repository lets it obtain the Space half itself. "Keep your Space check and layer this on top" is advice that consumer cannot act on, so the conjunction has to happen server-side.
The predicate comes from spacepkg.ActiveMembers rather than being spelled out again, so it cannot drift from CheckMembership's.
MembershipsInSpace is deliberately NOT changed, and the reason it used to give was false. It said the route "already ran SpaceMiddleware", so the consumer held the Space half. It does not: modules/user/api.go registers POST /v1/auth/verify in a group whose only middleware is the rate limiter, and the request carries a caller-supplied space_id.
The exemption survives on a different fact, which is the one to check if this is ever revisited: the uid is not caller-supplied. authVerifyToken derives it from the PRESENTED TOKEN (tokenValidator.Validate), so every answer this function can produce is about the token holder themselves, and the seat predicate means a `member: true` can only ever be self-information. A caller naming a Space they hold no seat in learns nothing, because they hold no seat in its projects either.
That is why it is exempt from the SPACE conjunction. It is NOT by itself a reason to exempt it from the ACTIVATION gate — see the note on that below.
Already-cached grants: closed elsewhere, not here ¶
This conjunction fixes FRESH answers only. A grant the consumer cached BEFORE the removal is invalidated by the epoch moving, and that bump is NOT in this function — it happens in the Space-removal transaction itself (modules/project.bumpMemberEpochForSpaceMemberTx, registered as a synchronous tx step).
It has to be there rather than here for a reason worth stating: closing the seats is asynchronous, the cleanup job can sit in backoff for minutes, and it has a terminal abandoned state after which nothing re-claims it. An invalidation signal that moves when the seat closes is therefore not bounded at all. Moving it at removal commit is what makes the peer contract's "epoch agreement means the cache is still good" true for this path.
This enumeration covers the project, seat and Space axes. It does NOT cover the ACCOUNT axis, and read top-to-bottom it used to look as if it did. A super-admin ban or an account destroy flips step 4's answer below while moving no epoch — the epoch is per PROJECT and the ban is per USER, so there is no per-project row for it to bump. Both directions ride epoch agreement unboundedly: a grant cached before the ban stays good, and a denial cached during it stays denied until some unrelated membership change happens to bump that project. The only bound on that axis is a time bound on the consumer's side; it is out of scope for this branch and specified in docs/project-membership-cache-bound-proposal.md, and the reason OCTO_MEMBERSHIP_INTERNAL_TOKEN stays unset until the peer implements it.
func RepairAbsentSentinelEpoch ¶
func RepairAbsentSentinelEpoch(ctx context.Context, session *dbr.Session, projectID string) (bool, error)
RepairAbsentSentinelEpoch lifts ONE active project off the reserved sentinel.
The `member_epoch = 0` predicate is its own CAS, so N replicas racing to repair the same row produce exactly one increment; and `status = 1` keeps it off disbanded rows, whose 0 is the correct answer. Increment-only, like every other writer of this column.
Reports whether a row was changed, so a caller can tell "repaired, retry is worth it" from "someone else already did, or the row is not actually anomalous".
This is a duplicate of modules/project's own repair statement, deliberately: that one is a private method on a module the read layer's consumers do not import, and the whole point here is that the endpoint that HIT the anomaly can fix it. The shared statement shape is pinned by the write-discipline guard, which scans for exactly this increment form. Takes a context because its only production caller runs it in a detached goroutine that holds an in-flight marker for this project id. On the process-wide session dbr reaches sql.DB with context.Background() and no Timeout, so both the pool acquisition and the statement were unbounded — and an unbounded one there does not merely leak a goroutine, it never releases the marker, which then rejects EVERY later repair attempt for that project for the life of the process. Recovery would fall back to the reconcile cursor, the hours-scale path this repair exists to avoid.
Tests may pass context.Background(): they are not holding a marker, and pinning a deadline is the caller's job.
func SetMembershipTearHookForTest ¶
func SetMembershipTearHookForTest(fn func()) (restore func(), err error)
SetMembershipTearHookForTest installs the hook above and returns a function that removes it.
Exported because the engine harness for this package's statements lives in modules/project — pkg/project is a predicate package with no database of its own, which is why its statements had no engine lane until that file was written. Refuses outside a test binary for the same reason modules/space's removal seams do: "nothing calls it" is a property of the current tree, not a boundary.
Types ¶
type Membership ¶
type Membership struct {
ProjectID string `db:"project_id"`
Role int `db:"role"`
MemberEpoch int64 `db:"member_epoch"`
}
Membership is one project's membership fact for a uid.
type ProjectMetadata ¶ added in v1.24.0
type ProjectMetadata struct {
ProjectID string `db:"project_id" json:"project_id"`
SpaceID string `db:"space_id" json:"space_id"`
Name string `db:"name" json:"name"`
Description string `db:"description" json:"description"`
Logo string `db:"logo" json:"logo"`
Role *int `json:"role"`
}
ProjectMetadata is the six-field projection returned by an internal lookup that has no user-role context. Role is deliberately nil and is encoded as JSON null; it must not be interpreted as a membership answer.
func ProjectMetadataInSpace ¶ added in v1.24.0
func ProjectMetadataInSpace(ctx context.Context, session *dbr.Session, spaceID string, projectIDs []string) ([]ProjectMetadata, error)
ProjectMetadataInSpace returns the matching active, activated Project rows without consulting any user or membership tables. The Space is resolved and checked in the same read-only REPEATABLE READ transaction as the Project read.
type SentinelAnomalyError ¶
type SentinelAnomalyError struct {
ProjectID string
}
SentinelAnomalyError carries WHICH project tripped the refusal, so a caller can repair that one row instead of waiting for the reconcile cursor to reach it.
The wait is the reason this type exists. The rollout document told operators the scan repairs such a row "within one rotation", but scanEpochSanity walks a bounded page budget per tick behind a PERSISTED cursor — and a row inserted by a not-yet-upgraded pod gets the highest id, so it is reached only when the cursor completes its current pass. On a large octo_project that is hours, and every request naming that id returns 500 for the whole window because the refusal is per-batch. RepairAbsentSentinelEpoch closes that to one indexed UPDATE.
func (*SentinelAnomalyError) Error ¶
func (e *SentinelAnomalyError) Error() string
func (*SentinelAnomalyError) Unwrap ¶
func (e *SentinelAnomalyError) Unwrap() error
Unwrap keeps errors.Is(err, ErrLiveProjectOnAbsentSentinel) true, which is what every existing caller branches on.
type UserProject ¶ added in v1.24.0
type UserProject struct {
ProjectID string `db:"project_id" json:"project_id"`
SpaceID string `db:"space_id" json:"space_id"`
Name string `db:"name" json:"name"`
Description string `db:"description" json:"description"`
Logo string `db:"logo" json:"logo"`
Role int `db:"role" json:"role"`
}
UserProject is one Project the target user holds an active seat in, reduced to the list's six fields: no epoch, counts or personal preference. role 0 is a real (member) value and is never omitted.
func UserProjectsInSpace ¶ added in v1.24.0
func UserProjectsInSpace(ctx context.Context, session *dbr.Session, spaceID, uid string) ([]UserProject, error)
UserProjectsInSpace returns every activated Project of spaceID that uid has joined, newest first (octo_project.id DESC, unpaginated); an empty non-nil slice means none.
A row qualifies only when the account is live (pkg/user.ActiveAccounts), the Space is active and uid holds an active seat in it (pkg/space.ActiveMembers), and the Project is status = 1 with activated_at set while uid's seat row is status = 1, removing = 0 and carries the Project's own space_id. All reads share one read-only REPEATABLE READ transaction.
The private seat UID and both Project/seat Space IDs are matched with ASCII-only FoldID before projection; SQL collation equality alone does not establish identity.
Legacy tables are reached only through pkg/user and pkg/space in separate legacy-only statements; joining them to octo_* mixes utf8mb4_0900_ai_ci with utf8mb4_general_ci and fails with MySQL error 1267 in production. Any failure returns (nil, err), never a partial or empty answer.