Repository navigation
feat(bigtable/bttest): honor the table's timestamp granularity - #20456
Conversation
The emulator accepted `Granularity` on CreateTable but discarded it, and
`validTimestamp` unconditionally required `ts % 1000 == 0`, so a MICROS
table behaved exactly like a MILLIS one.
Store the granularity on the table and validate against it:
- Server-generated timestamps (`TimestampMicros == -1`, i.e.
bigtable.ServerTime, and the ones ReadModifyWriteRow mints) are
truncated to the table's granularity rather than rejected, so a
MICROS table now keeps microsecond precision.
- User-provided timestamps that do not match the granularity are
rejected. This covers SetCell, DeleteFromColumn's TimeRange, and
AddToCell/MergeToCell -- the last two previously only checked
`ts >= 0`, despite the proto requiring the timestamp to match the
table's granularity.
- The upper bound is now per-granularity, so MICROS tables accept the
full int64 range instead of MaxInt64 truncated to a multiple of 1000.
Rejections now return InvalidArgument, matching the service, instead of
a bare error that surfaced as Unknown. CreateTable rejects unrecognized
granularity values, and GetTable/ModifyColumnFamilies report the table's
real granularity instead of always MILLIS.
The read path (timestamp_range_filter) is deliberately left unvalidated,
matching existing emulator behavior.
Change-Id: I3939df9812973240497b3830d56c0007cc0ad668
There was a problem hiding this comment.
Code Review
This pull request introduces support for table timestamp granularity (such as microsecond granularity) in the in-memory Bigtable test server (bttest). It updates table creation, retrieval, and modification APIs to track granularity, validates that incoming mutations align with the table's granularity, and adds comprehensive tests. The review feedback suggests improving error handling by returning structured gRPC InvalidArgument status errors instead of bare fmt.Errorf calls, correcting Go error string capitalization, and using the more idiomatic time.Now().UnixMicro() instead of manually dividing UnixNano() by 1e3.
| if ts < 0 { | ||
| return fmt.Errorf("AddToCell must set timestamp >= 0") | ||
| } |
There was a problem hiding this comment.
To align with the PR's goal of returning InvalidArgument instead of bare fmt.Errorf (which surfaces as Unknown to clients), this validation error should also return a gRPC status error with codes.InvalidArgument. Note that Go error strings should not be capitalized.
| if ts < 0 { | |
| return fmt.Errorf("AddToCell must set timestamp >= 0") | |
| } | |
| if ts < 0 { | |
| return status.Errorf(codes.InvalidArgument, "addToCell must set timestamp >= 0") | |
| } |
References
- Go error strings should not be capitalized and should not end with punctuation.
There was a problem hiding this comment.
Done, slightly differently: minValidMilliSeconds is 0, so validTimestamp on the next line already rejects ts < 0. Removed the redundant check instead of converting it — negative timestamps now fall through to invalidTimestampError, which returns InvalidArgument.
| if ts < 0 { | ||
| return fmt.Errorf("MergeToCell must set timestamp >= 0") | ||
| } |
There was a problem hiding this comment.
To maintain consistency and ensure invalid user-provided inputs return InvalidArgument instead of Unknown, convert this bare fmt.Errorf to a gRPC status error. Note that Go error strings should not be capitalized.
| if ts < 0 { | |
| return fmt.Errorf("MergeToCell must set timestamp >= 0") | |
| } | |
| if ts < 0 { | |
| return status.Errorf(codes.InvalidArgument, "mergeToCell must set timestamp >= 0") | |
| } |
References
- Go error strings should not be capitalized and should not end with punctuation.
There was a problem hiding this comment.
Same as the AddToCell case — removed the redundant ts < 0 check; validTimestamp/invalidTimestampError below already covers it and returns InvalidArgument.
| if tsr.StartTimestampMicros >= tsr.EndTimestampMicros && tsr.EndTimestampMicros != 0 { | ||
| return fmt.Errorf("inverted or invalid timestamp range [%d, %d]", tsr.StartTimestampMicros, tsr.EndTimestampMicros) |
There was a problem hiding this comment.
Convert this bare fmt.Errorf to a gRPC status error with codes.InvalidArgument to ensure the client receives a clear, structured error code.
| if tsr.StartTimestampMicros >= tsr.EndTimestampMicros && tsr.EndTimestampMicros != 0 { | |
| return fmt.Errorf("inverted or invalid timestamp range [%d, %d]", tsr.StartTimestampMicros, tsr.EndTimestampMicros) | |
| if tsr.StartTimestampMicros >= tsr.EndTimestampMicros && tsr.EndTimestampMicros != 0 { | |
| return status.Errorf(codes.InvalidArgument, "inverted or invalid timestamp range [%d, %d]", tsr.StartTimestampMicros, tsr.EndTimestampMicros) |
| func (t *table) newTimestamp() int64 { | ||
| ts := time.Now().UnixNano() / 1e3 | ||
| unit := t.timestampUnit() | ||
| return ts - ts%unit | ||
| } |
There was a problem hiding this comment.
In Go 1.17+, prefer using time.Now().UnixMicro() instead of manually dividing UnixNano() by 1e3 to get the current Unix time in microseconds. This is cleaner, more idiomatic, and avoids manual division.
| func (t *table) newTimestamp() int64 { | |
| ts := time.Now().UnixNano() / 1e3 | |
| unit := t.timestampUnit() | |
| return ts - ts%unit | |
| } | |
| func (t *table) newTimestamp() int64 { | |
| ts := time.Now().UnixMicro() | |
| unit := t.timestampUnit() | |
| return ts - ts%unit | |
| } |
|
|
||
| // On a millis table, the generated timestamp must be millis-aligned. | ||
| s, name := newGranularityServer(t, btapb.Table_MILLIS) | ||
| before := time.Now().UnixNano() / 1e3 |
| if _, err := s.MutateRow(ctx, setCell(name, -1)); err != nil { | ||
| t.Fatalf("MutateRow(ServerTime): %v", err) | ||
| } | ||
| after := time.Now().UnixNano() / 1e3 |
Change-Id: I4ae6db7d6ab8c46e13dd9ae581e6ad9d4d980b2b
🤖 I have created a release *beep* *boop* --- ## [1.55.0](googleapis/google-cloud-go@bigtable/v1.54.0...bigtable/v1.55.0) (2026-09-03) ### Features * **bigtable/bttest:** Honor the table's timestamp granularity ([googleapis#20456](googleapis#20456)) ([0054866](googleapis@0054866)) * **bigtable:** Debug tags for GetClientConfiguration session_configuration state ([googleapis#20472](googleapis#20472)) ([8e57e4b](googleapis@8e57e4b)) ### Bug Fixes * **bigtable:** Use instance_name instead of name in GetClientConfig ([googleapis#20465](googleapis#20465)) ([375f198](googleapis@375f198)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
🤖 I have created a release *beep* *boop* --- ## [1.55.0](bigtable/v1.54.0...bigtable/v1.55.0) (2026-09-03) ### Features * **bigtable/bttest:** Honor the table's timestamp granularity ([#20456](#20456)) ([0054866](0054866)) * **bigtable:** Debug tags for GetClientConfiguration session_configuration state ([#20472](#20472)) ([8e57e4b](8e57e4b)) ### Bug Fixes * **bigtable:** Use instance_name instead of name in GetClientConfig ([#20465](#20465)) ([375f198](375f198)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Follows the principle #20456 established for `TimestampMicros == -1` — "Server-generated timestamps ... are truncated to the table's granularity rather than rejected" — and applies it to the second generated case, which `data.proto` defines explicitly: - `USER_SPECIFIED` — "The server will **reject** the mutation if the timestamp's precision does not match the table's granularity." - `CLIENT_AUTO_GENERATED` — "The server will **truncate** the timestamp's precision to match the table's granularity." `bttest` did not read `Mutation.timestamp_origin` at all, so the second rule was not implemented. Measured both ends, 2026-09-03 (details in #20468): | Target | Before | After | |---|---|---| | Bigtable, `MILLIS` table | accepted, stored truncated | unchanged | | `bttest`, `MILLIS` table | `InvalidArgument: invalid timestamp …` | accepted, stored truncated | `applyMutations` is shared by `MutateRow`, `MutateRows` and `CheckAndMutateRow`, so one place covers all three. `AddToCell` and `MergeToCell` are left alone: their timestamps are user-supplied by construction. The new test fails on exactly one case against the unpatched code — `millis table truncates a client-generated micros timestamp` — and passes on the other four, which encode behaviour that already existed. Fixes #20468
Problem
The emulator accepted
GranularityonCreateTablebut discarded it, andvalidTimestampunconditionally requiredts % 1000 == 0. AMICROStable therefore behaved exactly like aMILLISone: microsecond-precision writes were rejected, andGetTablereported the wrong granularity.Change
tablenow carries the granularity (unspecified →MILLIS, immutable after creation), and timestamps are validated against it:TimestampMicros == -1(bigtable.ServerTime) and the timestampsReadModifyWriteRowmints go through(*table).newTimestamp(), which truncates to the table's granularity — so aMICROStable keeps microsecond precision where aMILLIStable still rounds down.SetCell,DeleteFromColumn'sTimeRange, andAddToCell/MergeToCell. The latter two previously only checkedts >= 0, despite the proto stating the timestamp "must be araw_timestamp_microsthat matches the table'sgranularity".MICROStables accept the fullint64range instead ofMaxInt64truncated to a multiple of 1000.Two smaller correctness fixes came along with it:
InvalidArgument(matching the service) instead of a barefmt.Errorfthat surfaced asUnknown, and a granularity mismatch gets a distinct message from an out-of-range value.CreateTablerejects unrecognized granularity enum values, andGetTable/ModifyColumnFamiliesreport the table's real granularity instead of hard-codingMILLIS.Deliberately out of scope
The read path (
timestamp_range_filter) is still not validated against granularity, matching existing emulator behavior — tightening it would break tests that pass arbitrary microsecond filters.UpdateTablestill can't change granularity, matching the real service.Testing
New coverage in
bttest/timestamp_test.go: accept/reject across both granularities, the max-timestamp bound per granularity, server-time truncation,DeleteFromColumnranges, admin round-tripping, and unknown-granularity rejection.go build ./...,go vet ./bttest/, andgo test ./bttest/all pass. The remaining failures in the widerbigtablemodule (TestConformance,TestIntegration_SessionVRpc_DeadlineExceeded,TestTableImpl_BypassesDivertibleGate) are pre-existing and reproduce identically on the base commit.