Documentation
¶
Index ¶
- Constants
- Variables
- func ConfigureConnReadDeadline(poolConfig *pgxpool.Config)
- func DataSourceName(d DataSourceNameParams) (string, error)
- func DecodeSnapshotState(raw []byte) (*committerpb.SnapshotState, error)
- func EncodeSnapshotState(state *committerpb.SnapshotState) ([]byte, error)
- func FmtNsID(sqlTemplate, namespaceID string) string
- func GetTablePreSplitTablets(ctx context.Context, pg *pgxpool.Pool, config *Config) (int, error)
- func IsYugabyteDB(ctx context.Context, pool *pgxpool.Pool) (bool, error)
- func MakeNsTablesQuery(nsID string, tablets int) string
- func MarkSnapshotCheckpointedInTx(ctx context.Context, tx pgx.Tx, blockNum uint64) error
- func NewPool(ctx context.Context, config *Config) (*pgxpool.Pool, error)
- func SetupSystemTablesAndNamespaces(ctx context.Context, config *Config) error
- func TableName(nsID string) string
- type Config
- type DataSourceNameParams
- type SnapshotStateManager
- type SnapshotUpdate
- type TLSConfig
Constants ¶
const (
// TxStatusTableName is the name of the system table holding per-transaction status.
TxStatusTableName = "tx_status"
)
Variables ¶
var ( // ErrDatabaseNotFound reports that a pool was opened against a database that does // not exist (SQLSTATE 3D000, invalid_catalog_name). It is exposed so a caller whose // database is guaranteed to exist can tell this apart from a connection failure and // treat it as the invariant violation it is, rather than retrying it. ErrDatabaseNotFound = errors.New("database does not exist") )
var ErrUnexpectedSnapshotStatus = errors.New("unexpected _snapshot record status")
ErrUnexpectedSnapshotStatus reports that the locked `_snapshot` record was not in a status its caller required, so the update was not applied. It is not a fault: a concurrent checkpoint reaches it legitimately. It is never retryable, because the record has moved on -- re-reading cannot restore the caller's premise, and the caller must re-decide what to do from the new status.
var LatestSnapshotPointerKey = []byte("latest snapshot key")
LatestSnapshotPointerKey is the `metadata`-table key whose value is the tx_id of the most recently accepted `_snapshot` record, so a reader can look up that record's current status with a single key lookup instead of scanning the (small but growing) ns__snapshot table. Pre-seeded (NULL) at DB init and written atomically, in the same DB transaction as the `_snapshot` row it points to, by the validator-committer's commit path.
Functions ¶
func ConfigureConnReadDeadline ¶
ConfigureConnReadDeadline configures pool connections with a per-read deadline so that reads on dead connections fail instead of hanging indefinitely.
We use a custom DialFunc that wraps every net.Conn with deadlineConn, which calls SetReadDeadline before each Read. Pool callbacks like AfterConnect or BeforeAcquire only fire once, so they can only set a single absolute deadline that expires even if the connection is healthy. Wrapping net.Conn lets us reset the deadline on every Read call.
The read timeout is set to 60 seconds, which is more than enough for read-set validation and commit operations. This can be made configurable if needed.
On macOS, Docker Desktop runs a userspace TCP proxy inside a Linux VM. When a container dies the proxy does not immediately close forwarded connections, so reads hang indefinitely. The deadlineConn wrapper ensures stuck reads fail within the timeout, allowing the pool to replace them. On Linux, reads already fail immediately via TCP RST when a node dies, so the wrapper is harmless.
func DataSourceName ¶
func DataSourceName(d DataSourceNameParams) (string, error)
DataSourceName returns the data source name of the database.
func DecodeSnapshotState ¶ added in v1.0.6
func DecodeSnapshotState(raw []byte) (*committerpb.SnapshotState, error)
DecodeSnapshotState unmarshals a `_snapshot` record value.
func EncodeSnapshotState ¶ added in v1.0.6
func EncodeSnapshotState(state *committerpb.SnapshotState) ([]byte, error)
EncodeSnapshotState marshals a `_snapshot` record value.
func FmtNsID ¶
FmtNsID replaces the namespace placeholder with the namespace ID in an SQL template string.
func GetTablePreSplitTablets ¶
GetTablePreSplitTablets determines the number of tablets to use for table pre-splitting. Returns 0 for PostgreSQL or when TablePreSplitTablets is not configured. Returns the configured value for YugabyteDB.
func IsYugabyteDB ¶
IsYugabyteDB queries the database version string to determine whether the backend is YugabyteDB. YugabyteDB's version() output contains "-YB-" (e.g., "PostgreSQL 11.2-YB-2.20.1.0 ..."), which distinguishes it from standard PostgreSQL.
func MakeNsTablesQuery ¶
MakeNsTablesQuery returns the query to create the table and functions for a namespace.
func MarkSnapshotCheckpointedInTx ¶ added in v1.0.6
MarkSnapshotCheckpointedInTx marks the latest snapshot CHECKPOINTED in the caller's transaction. The checkpoint write and this update must commit together. Otherwise, a crash could leave the snapshot waiting for a checkpoint and block new snapshots.
An already-CHECKPOINTED record is left unchanged. A missing pointer or a different block returns a non-retryable error: the record no longer matches what was verified.
func NewPool ¶
NewPool creates a new pool from a database config.
A pool open against a database that does not exist fails immediately rather than being retried: the database is provisioned before any service starts, so SQLSTATE 3D000 means the configured name is wrong or the cluster was never initialized, and no number of attempts changes either. Retrying it would spend the whole retry budget before reporting a condition that was decidable on the first try.
func SetupSystemTablesAndNamespaces ¶
SetupSystemTablesAndNamespaces creates the required system tables and namespaces in the database. This function should be called before operating against the database. It is safe to run multiple times.
Types ¶
type Config ¶
type Config struct {
Endpoints []*connection.Endpoint `mapstructure:"endpoints" default:"localhost:5433"`
Username string `mapstructure:"username"`
Password string `mapstructure:"password"`
Database string `mapstructure:"database" default:"yugabyte"`
MaxConnections int32 `mapstructure:"max-connections" default:"20" validate:"gt=0"`
MinConnections int32 `mapstructure:"min-connections" default:"1" validate:"gte=0"`
LoadBalance bool `mapstructure:"load-balance"`
Retry *retry.Profile `mapstructure:"retry" default:"max-elapsed-time=10m"`
TLS TLSConfig `mapstructure:"tls"`
TablePreSplitTablets int `mapstructure:"table-pre-split-tablets"`
}
Config is the configuration for database initialization and connection. This configuration is shared across multiple services (VC, Query) and the init-db CLI command.
func (*Config) DataSourceName ¶
DataSourceName returns the data source name of the database.
func (*Config) EndpointsString ¶
EndpointsString returns the address:port as a string with comma as a separator between endpoints.
type DataSourceNameParams ¶
type DataSourceNameParams struct {
Username string
Password string
Database string
EndpointsString string
LoadBalance bool
TLS TLSConfig
}
DataSourceNameParams defines the parameters required to construct a database connection string.
type SnapshotStateManager ¶ added in v1.0.6
type SnapshotStateManager struct {
// contains filtered or unexported fields
}
SnapshotStateManager reads and writes `_snapshot` records over a state-database pool. Callers hold one per process; it carries no per-record state, so it is safe to share.
func NewSnapshotStateManager ¶ added in v1.0.6
func NewSnapshotStateManager(pool *pgxpool.Pool, retryProfile *retry.Profile) *SnapshotStateManager
NewSnapshotStateManager returns a SnapshotStateManager over an already-open state-database pool. The caller keeps ownership of the pool, including closing it.
func (*SnapshotStateManager) ReadLatest ¶ added in v1.0.6
func (s *SnapshotStateManager) ReadLatest(ctx context.Context) (*committerpb.SnapshotState, error)
ReadLatest performs the full pointer-to-row read cycle: it looks up the latest-record pointer (LatestSnapshotPointerKey) and, when one is set, reads and decodes the `_snapshot` record it names.
Returns (nil, nil) when no snapshot has ever been accepted (pointer unset).
A pointer that names a missing row, and a row whose value does not decode, are both hard errors rather than "no snapshot": the pointer is written in the same DB transaction as the row it names, so either state is corruption. Reporting it as absent would let the validator-committer conclude no snapshot is in flight and accept a new one. Both are also non-retryable, because neither a missing row nor an undecodable value can resolve itself on a later attempt.
func (*SnapshotStateManager) Update ¶ added in v1.0.6
func (s *SnapshotStateManager) Update(ctx context.Context, ref *committerpb.TxRef, update SnapshotUpdate) error
Update rewrites the `_snapshot` record for ref.TxId per update; TxRef and CloneDatabase are preserved because the existing record is decoded, mutated, and re-encoded rather than rebuilt.
The read and the write run inside a single DB transaction using SELECT ... FOR UPDATE (see selectSnapshotRecordForUpdateSQL), not READ COMMITTED alone: without the row lock a concurrent writer could commit between our SELECT and UPDATE, and we would still overwrite it with our stale re-encoded value, with no error at any point. The whole read-decode-mutate-encode-write sequence is retried as one unit, so a transient failure anywhere in it restarts from a fresh, consistent read.
The row lock stops a lost update; it does not stop a wrong-state transition, since this UPDATE matches on `key` alone and would apply whatever the locked row now holds. update.ExpectedStatus closes that: when set, the locked status must be one of the statuses the caller reasoned about, or the update is rejected with ErrUnexpectedSnapshotStatus. An empty ExpectedStatus keeps the write unconditional.
type SnapshotUpdate ¶ added in v1.0.6
type SnapshotUpdate struct {
Status committerpb.SnapshotState_Status
Digest []byte
ErrMsg string
// ExpectedStatus, when non-empty, aborts the update unless the locked row's status
// is one of these; an empty slice writes unconditionally. See Update for why the row
// lock alone does not give this.
ExpectedStatus []committerpb.SnapshotState_Status
}
SnapshotUpdate bundles the fields SnapshotStateManager.Update can change. Status is required. A nil Digest keeps the record's existing Hash, because a status move that publishes no new digest (IN_PROGRESS) must not wipe one.
ErrMsg, by contrast, is written unconditionally: the diagnostic describes the status it arrives with, so a record that reaches COMPLETED after a failed attempt must not keep the failed attempt's error text, which would otherwise read as a completed snapshot that also failed.
Source Files