Skip to content

feat(bigtable/bttest): honor the table's timestamp granularity - #20456

Merged
mutianf merged 2 commits into
googleapis:mainfrom
mutianf:bigtable-emulator-micros-granularity
Sep 3, 2026
Merged

mutianf merged 2 commits into
googleapis:mainfrom
mutianf:bigtable-emulator-micros-granularity

Conversation

@mutianf

@mutianf mutianf commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Problem

The emulator accepted Granularity on CreateTable but discarded it, and validTimestamp unconditionally required ts % 1000 == 0. A MICROS table therefore behaved exactly like a MILLIS one: microsecond-precision writes were rejected, and GetTable reported the wrong granularity.

Change

table now carries the granularity (unspecified → MILLIS, immutable after creation), and timestamps are validated against it:

  • Server-generated timestamps are truncated, not rejected. TimestampMicros == -1 (bigtable.ServerTime) and the timestamps ReadModifyWriteRow mints go through (*table).newTimestamp(), which truncates to the table's granularity — so a MICROS table keeps microsecond precision where a MILLIS table still rounds down.
  • User-provided timestamps that don't match the granularity are rejected. This covers SetCell, DeleteFromColumn's TimeRange, and AddToCell/MergeToCell. The latter two previously only checked ts >= 0, despite the proto stating the timestamp "must be a raw_timestamp_micros that matches the table's granularity".
  • The upper bound is per-granularity. MICROS tables accept the full int64 range instead of MaxInt64 truncated to a multiple of 1000.

Two smaller correctness fixes came along with it:

  • Rejections return InvalidArgument (matching the service) instead of a bare fmt.Errorf that surfaced as Unknown, and a granularity mismatch gets a distinct message from an out-of-range value.
  • CreateTable rejects unrecognized granularity enum values, and GetTable/ModifyColumnFamilies report the table's real granularity instead of hard-coding MILLIS.

Deliberately out of scope

The read path (timestamp_range_filter) is still not validated against granularity, matching existing emulator behavior — tightening it would break tests that pass arbitrary microsecond filters. UpdateTable still can't change granularity, matching the real service.

Testing

New coverage in bttest/timestamp_test.go: accept/reject across both granularities, the max-timestamp bound per granularity, server-time truncation, DeleteFromColumn ranges, admin round-tripping, and unknown-granularity rejection.

go build ./..., go vet ./bttest/, and go test ./bttest/ all pass. The remaining failures in the wider bigtable module (TestConformance, TestIntegration_SessionVRpc_DeadlineExceeded, TestTableImpl_BypassesDivertibleGate) are pre-existing and reproduce identically on the base commit.

The emulator accepted `Granularity` on CreateTable but discarded it, and
`validTimestamp` unconditionally required `ts % 1000 == 0`, so a MICROS
table behaved exactly like a MILLIS one.

Store the granularity on the table and validate against it:

  - Server-generated timestamps (`TimestampMicros == -1`, i.e.
    bigtable.ServerTime, and the ones ReadModifyWriteRow mints) are
    truncated to the table's granularity rather than rejected, so a
    MICROS table now keeps microsecond precision.
  - User-provided timestamps that do not match the granularity are
    rejected. This covers SetCell, DeleteFromColumn's TimeRange, and
    AddToCell/MergeToCell -- the last two previously only checked
    `ts >= 0`, despite the proto requiring the timestamp to match the
    table's granularity.
  - The upper bound is now per-granularity, so MICROS tables accept the
    full int64 range instead of MaxInt64 truncated to a multiple of 1000.

Rejections now return InvalidArgument, matching the service, instead of
a bare error that surfaced as Unknown. CreateTable rejects unrecognized
granularity values, and GetTable/ModifyColumnFamilies report the table's
real granularity instead of always MILLIS.

The read path (timestamp_range_filter) is deliberately left unvalidated,
matching existing emulator behavior.

Change-Id: I3939df9812973240497b3830d56c0007cc0ad668
@mutianf
mutianf requested review from a team as code owners September 2, 2026 15:54
@product-auto-label product-auto-label Bot added the api: bigtable Issues related to the Bigtable API. label Sep 2, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for table timestamp granularity (such as microsecond granularity) in the in-memory Bigtable test server (bttest). It updates table creation, retrieval, and modification APIs to track granularity, validates that incoming mutations align with the table's granularity, and adds comprehensive tests. The review feedback suggests improving error handling by returning structured gRPC InvalidArgument status errors instead of bare fmt.Errorf calls, correcting Go error string capitalization, and using the more idiomatic time.Now().UnixMicro() instead of manually dividing UnixNano() by 1e3.

Comment thread bigtable/bttest/inmem.go Outdated
Comment on lines 1226 to 1228
if ts < 0 {
return fmt.Errorf("AddToCell must set timestamp >= 0")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To align with the PR's goal of returning InvalidArgument instead of bare fmt.Errorf (which surfaces as Unknown to clients), this validation error should also return a gRPC status error with codes.InvalidArgument. Note that Go error strings should not be capitalized.

Suggested change
if ts < 0 {
return fmt.Errorf("AddToCell must set timestamp >= 0")
}
if ts < 0 {
return status.Errorf(codes.InvalidArgument, "addToCell must set timestamp >= 0")
}
References
  1. Go error strings should not be capitalized and should not end with punctuation.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, slightly differently: minValidMilliSeconds is 0, so validTimestamp on the next line already rejects ts < 0. Removed the redundant check instead of converting it — negative timestamps now fall through to invalidTimestampError, which returns InvalidArgument.

Comment thread bigtable/bttest/inmem.go Outdated
Comment on lines 1259 to 1261
if ts < 0 {
return fmt.Errorf("MergeToCell must set timestamp >= 0")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To maintain consistency and ensure invalid user-provided inputs return InvalidArgument instead of Unknown, convert this bare fmt.Errorf to a gRPC status error. Note that Go error strings should not be capitalized.

Suggested change
if ts < 0 {
return fmt.Errorf("MergeToCell must set timestamp >= 0")
}
if ts < 0 {
return status.Errorf(codes.InvalidArgument, "mergeToCell must set timestamp >= 0")
}
References
  1. Go error strings should not be capitalized and should not end with punctuation.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the AddToCell case — removed the redundant ts < 0 check; validTimestamp/invalidTimestampError below already covers it and returns InvalidArgument.

Comment thread bigtable/bttest/inmem.go Outdated
Comment on lines 1299 to 1300
if tsr.StartTimestampMicros >= tsr.EndTimestampMicros && tsr.EndTimestampMicros != 0 {
return fmt.Errorf("inverted or invalid timestamp range [%d, %d]", tsr.StartTimestampMicros, tsr.EndTimestampMicros)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Convert this bare fmt.Errorf to a gRPC status error with codes.InvalidArgument to ensure the client receives a clear, structured error code.

Suggested change
if tsr.StartTimestampMicros >= tsr.EndTimestampMicros && tsr.EndTimestampMicros != 0 {
return fmt.Errorf("inverted or invalid timestamp range [%d, %d]", tsr.StartTimestampMicros, tsr.EndTimestampMicros)
if tsr.StartTimestampMicros >= tsr.EndTimestampMicros && tsr.EndTimestampMicros != 0 {
return status.Errorf(codes.InvalidArgument, "inverted or invalid timestamp range [%d, %d]", tsr.StartTimestampMicros, tsr.EndTimestampMicros)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread bigtable/bttest/inmem.go
Comment on lines +1697 to 1701
func (t *table) newTimestamp() int64 {
ts := time.Now().UnixNano() / 1e3
unit := t.timestampUnit()
return ts - ts%unit
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In Go 1.17+, prefer using time.Now().UnixMicro() instead of manually dividing UnixNano() by 1e3 to get the current Unix time in microseconds. This is cleaner, more idiomatic, and avoids manual division.

Suggested change
func (t *table) newTimestamp() int64 {
ts := time.Now().UnixNano() / 1e3
unit := t.timestampUnit()
return ts - ts%unit
}
func (t *table) newTimestamp() int64 {
ts := time.Now().UnixMicro()
unit := t.timestampUnit()
return ts - ts%unit
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread bigtable/bttest/timestamp_test.go Outdated

// On a millis table, the generated timestamp must be millis-aligned.
s, name := newGranularityServer(t, btapb.Table_MILLIS)
before := time.Now().UnixNano() / 1e3

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use time.Now().UnixMicro() instead of manually dividing UnixNano() by 1e3 for cleaner and more idiomatic Go code.

Suggested change
before := time.Now().UnixNano() / 1e3
before := time.Now().UnixMicro()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread bigtable/bttest/timestamp_test.go Outdated
if _, err := s.MutateRow(ctx, setCell(name, -1)); err != nil {
t.Fatalf("MutateRow(ServerTime): %v", err)
}
after := time.Now().UnixNano() / 1e3

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use time.Now().UnixMicro() instead of manually dividing UnixNano() by 1e3 for cleaner and more idiomatic Go code.

Suggested change
after := time.Now().UnixNano() / 1e3
after := time.Now().UnixMicro()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Change-Id: I4ae6db7d6ab8c46e13dd9ae581e6ad9d4d980b2b
@mutianf
mutianf merged commit 0054866 into googleapis:main Sep 3, 2026
19 checks passed
@mutianf
mutianf deleted the bigtable-emulator-micros-granularity branch September 3, 2026 03:25
machi-genggeng pushed a commit to machi-genggeng/google-cloud-go that referenced this pull request Sep 3, 2026
🤖 I have created a release *beep* *boop*
---


##
[1.55.0](googleapis/google-cloud-go@bigtable/v1.54.0...bigtable/v1.55.0)
(2026-09-03)


### Features

* **bigtable/bttest:** Honor the table's timestamp granularity
([googleapis#20456](googleapis#20456))
([0054866](googleapis@0054866))
* **bigtable:** Debug tags for GetClientConfiguration
session_configuration state
([googleapis#20472](googleapis#20472))
([8e57e4b](googleapis@8e57e4b))


### Bug Fixes

* **bigtable:** Use instance_name instead of name in GetClientConfig
([googleapis#20465](googleapis#20465))
([375f198](googleapis@375f198))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
sushanb pushed a commit that referenced this pull request Sep 3, 2026
🤖 I have created a release *beep* *boop*
---


##
[1.55.0](bigtable/v1.54.0...bigtable/v1.55.0)
(2026-09-03)


### Features

* **bigtable/bttest:** Honor the table's timestamp granularity
([#20456](#20456))
([0054866](0054866))
* **bigtable:** Debug tags for GetClientConfiguration
session_configuration state
([#20472](#20472))
([8e57e4b](8e57e4b))


### Bug Fixes

* **bigtable:** Use instance_name instead of name in GetClientConfig
([#20465](#20465))
([375f198](375f198))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
mutianf pushed a commit that referenced this pull request Sep 8, 2026
Follows the principle #20456 established for `TimestampMicros == -1` —
"Server-generated
timestamps ... are truncated to the table's granularity rather than
rejected" — and applies it to
the second generated case, which `data.proto` defines explicitly:

- `USER_SPECIFIED` — "The server will **reject** the mutation if the
timestamp's precision does not
  match the table's granularity."
- `CLIENT_AUTO_GENERATED` — "The server will **truncate** the
timestamp's precision to match the
  table's granularity."

`bttest` did not read `Mutation.timestamp_origin` at all, so the second
rule was not implemented.

Measured both ends, 2026-09-03 (details in #20468):

| Target | Before | After |
|---|---|---|
| Bigtable, `MILLIS` table | accepted, stored truncated | unchanged |
| `bttest`, `MILLIS` table | `InvalidArgument: invalid timestamp …` |
accepted, stored truncated |

`applyMutations` is shared by `MutateRow`, `MutateRows` and
`CheckAndMutateRow`, so one place
covers all three. `AddToCell` and `MergeToCell` are left alone: their
timestamps are user-supplied
by construction.

The new test fails on exactly one case against the unpatched code —
`millis table truncates a
client-generated micros timestamp` — and passes on the other four, which
encode behaviour that
already existed.

Fixes #20468
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the Bigtable API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

⌂ 换网址