Documentation
¶
Overview ¶
Package blobpool implements the EIP-4844 blob transaction pool.
Index ¶
- Variables
- type BlobBuffer
- func (b *BlobBuffer) AddCells(hash common.Hash, deliveries map[string]*PeerDelivery, ...)
- func (b *BlobBuffer) AddTx(txs []*types.Transaction, peer string) []error
- func (b *BlobBuffer) Flush() ([]common.Hash, []error)
- func (b *BlobBuffer) HasCells(hash common.Hash) bool
- func (b *BlobBuffer) HasTx(hash common.Hash) bool
- type BlobBufferFunctions
- type BlobPool
- func (p *BlobPool) Add(txs []*types.Transaction, sync bool) []error
- func (p *BlobPool) AddPooledTx(ptx *BlobTxForPool) (err error)
- func (p *BlobPool) Clear()
- func (p *BlobPool) Close() error
- func (p *BlobPool) Content() (map[common.Address][]*types.Transaction, ...)
- func (p *BlobPool) ContentFrom(addr common.Address) ([]*types.Transaction, []*types.Transaction)
- func (p *BlobPool) Filter(tx *types.Transaction) bool
- func (p *BlobPool) FilterType(kind byte) bool
- func (p *BlobPool) Get(hash common.Hash) *types.Transaction
- func (p *BlobPool) GetBlobCells(vhashes []common.Hash, mask types.CustodyBitmap) ([][]*kzg4844.Cell, [][]*kzg4844.Proof, error)
- func (p *BlobPool) GetBlobHashes(txHash common.Hash) []common.Hash
- func (p *BlobPool) GetCustody(hash common.Hash) *types.CustodyBitmap
- func (p *BlobPool) GetMetadata(hash common.Hash) *txpool.TxMetadata
- func (p *BlobPool) GetRLP(hash common.Hash, version uint) []byte
- func (p *BlobPool) Has(hash common.Hash) bool
- func (p *BlobPool) Init(gasTip uint64, head *types.Header, reserver txpool.Reserver) error
- func (p *BlobPool) Nonce(addr common.Address) uint64
- func (p *BlobPool) Pending(filter txpool.PendingFilter) (map[common.Address][]*txpool.LazyTransaction, int)
- func (p *BlobPool) Reset(oldHead, newHead *types.Header)
- func (p *BlobPool) SetGasTip(tip *big.Int)
- func (p *BlobPool) Stats() (int, int)
- func (p *BlobPool) Status(hash common.Hash) txpool.TxStatus
- func (p *BlobPool) SubscribeTransactions(ch chan<- core.NewTxsEvent, reorgs bool) event.Subscription
- func (p *BlobPool) ValidateTxBasics(tx *types.Transaction) error
- type BlobTxForPool
- type BlockChain
- type Cache
- func (c *Cache) GetBlobs(ctx context.Context, vhashes []common.Hash, version byte) (_ []*kzg4844.Blob, _ []kzg4844.Commitment, _ [][]kzg4844.Proof, err error)
- func (c *Cache) GetCells(vhashes []common.Hash, mask types.CustodyBitmap) ([][]*kzg4844.Cell, [][]*kzg4844.Proof, error)
- func (c *Cache) HasBlobs(ctx context.Context, vhashes []common.Hash) []bool
- func (c *Cache) SetCellMode(on bool)
- func (c *Cache) Stop()
- type Config
- type PeerDelivery
Constants ¶
This section is empty.
Variables ¶
var DefaultConfig = Config{
Datadir: "blobpool",
Datacap: 10 * 1024 * 1024 * 1024 / 4,
PriceBump: 100,
BlockedRatio: 0.5,
}
DefaultConfig contains the default configurations for the transaction pool.
Functions ¶
This section is empty.
Types ¶
type BlobBuffer ¶ added in v1.17.5
type BlobBuffer struct {
// contains filtered or unexported fields
}
func NewBlobBuffer ¶ added in v1.17.5
func NewBlobBuffer(cb BlobBufferFunctions) *BlobBuffer
func (*BlobBuffer) AddCells ¶ added in v1.17.5
func (b *BlobBuffer) AddCells(hash common.Hash, deliveries map[string]*PeerDelivery, custody types.CustodyBitmap)
AddCells buffers per-peer cell deliveries from the blob fetcher. If the transaction is already buffered, verification and pool insertion are attempted.
func (*BlobBuffer) AddTx ¶ added in v1.17.5
func (b *BlobBuffer) AddTx(txs []*types.Transaction, peer string) []error
AddTx buffers a blob transaction (without blobs) from an ETH/72 peer. If cells are already buffered, verification and pool insertion are attempted.
type BlobBufferFunctions ¶ added in v1.17.5
type BlobBufferFunctions struct {
ValidateTx func(*types.Transaction) error
AddToPool func(*BlobTxForPool) error
DropPeer func(string)
}
type BlobPool ¶
type BlobPool struct {
// contains filtered or unexported fields
}
BlobPool is the transaction pool dedicated to EIP-4844 blob transactions.
Blob transactions are special snowflakes that are designed for a very specific purpose (rollups) and are expected to adhere to that specific use case. These behavioural expectations allow us to design a transaction pool that is more robust (i.e. resending issues) and more resilient to DoS attacks (e.g. replace-flush attacks) than the generic tx pool. These improvements will also mean, however, that we enforce a significantly more aggressive strategy on entering and exiting the pool:
Blob transactions are large. With the initial design aiming for 128KB blobs, we must ensure that these only traverse the network the absolute minimum number of times. Broadcasting to sqrt(peers) is out of the question, rather these should only ever be announced and the remote side should request it if it wants to.
Block blob-space is limited. With blocks being capped to a few blob txs, we can make use of the very low expected churn rate within the pool. Notably, we should be able to use a persistent disk backend for the pool, solving the tx resend issue that plagues the generic tx pool, as long as there's no artificial churn (i.e. pool wars).
Purpose of blobs are layer-2s. Layer-2s are meant to use blob transactions to commit to their own current state, which is independent of Ethereum mainnet (state, txs). This means that there's no reason for blob tx cancellation or replacement, apart from a potential basefee / miner tip adjustment.
Replacements are expensive. Given their size, propagating a replacement blob transaction to an existing one should be aggressively discouraged. Whilst generic transactions can start at 1 Wei gas cost and require a 10% fee bump to replace, we suggest requiring a higher min cost (e.g. 1 gwei) and a more aggressive bump (100%).
Cancellation is prohibitive. Evicting an already propagated blob tx is a huge DoS vector. As such, a) replacement (higher-fee) blob txs mustn't invalidate already propagated (future) blob txs (cumulative fee); b) nonce-gapped blob txs are disallowed; c) the presence of blob transactions exclude non-blob transactions.
Malicious cancellations are possible. Although the pool might prevent txs that cancel blobs, blocks might contain such transaction (malicious miner or flashbotter). The pool should cap the total number of blob transactions per account as to prevent propagating too much data before cancelling it via a normal transaction. It should nonetheless be high enough to support resurrecting reorged transactions. Perhaps 4-16.
It is not the role of the blobpool to serve as a storage for limit orders below market: blob transactions with fee caps way below base fee or blob fee. Therefore, the propagation of blob transactions that are far from being includable is suppressed. The pool will only announce blob transactions that are close to being includable (based on the current fees and the transaction's fee caps), and will delay the announcement of blob transactions that are far from being includable until base fee and/or blob fee is reduced.
Local txs are meaningless. Mining pools historically used local transactions for payouts or for backdoor deals. With 1559 in place, the basefee usually dominates the final price, so 0 or non-0 tip doesn't change much. Blob txs retain the 1559 2D gas pricing (and introduce on top a dynamic blob gas fee), so locality is moot. With a disk backed blob pool avoiding the resend issue, there's also no need to save own transactions for later.
No-blob blob-txs are bad. Theoretically there's no strong reason to disallow blob txs containing 0 blobs. In practice, admitting such txs into the pool breaks the low-churn invariant as blob constraints don't apply anymore. Even though we could accept blocks containing such txs, a reorg would require moving them back into the blob pool, which can break invariants.
Dropping blobs needs delay. When normal transactions are included, they are immediately evicted from the pool since they are contained in the including block. Blobs however are not included in the execution chain, so a mini reorg cannot re-pool "lost" blob transactions. To support reorgs, blobs are retained on disk until they are finalised.
Blobs can arrive via flashbots. Blocks might contain blob transactions we have never seen on the network. Since we cannot recover them from blocks either, the engine_newPayload needs to give them to us, and we cache them until finality to support reorgs without tx losses.
Whilst some constraints above might sound overly aggressive, the general idea is that the blob pool should work robustly for its intended use case and whilst anyone is free to use blob transactions for arbitrary non-rollup use cases, they should not be allowed to run amok the network.
Implementation wise there are a few interesting design choices:
Adding a transaction to the pool blocks until persisted to disk. This is viable because TPS is low (2-4 blobs per block initially, maybe 8-16 at peak), so natural churn is a couple MB per block. Replacements doing O(n) updates are forbidden and transaction propagation is pull based (i.e. no pileup of pending data).
When transactions are chosen for inclusion, the primary criteria is the signer tip (and having a basefee/data fee high enough of course). However, same-tip transactions will be split by their basefee/datafee, preferring those that are closer to the current network limits. The idea being that very relaxed ones can be included even if the fees go up, when the closer ones could already be invalid.
Because the maximum number of blobs allowed in a block can change per fork, the pool is designed to handle the maximum number of blobs allowed in the chain's latest defined fork -- even if it isn't active. This avoids needing to upgrade the database around the fork boundary.
When the pool eventually reaches saturation, some old transactions - that may never execute - will need to be evicted in favor of newer ones. The eviction strategy is quite complex:
Exceeding capacity evicts the highest-nonce of the account with the lowest paying blob transaction anywhere in the pooled nonce-sequence, as that tx would be executed the furthest in the future and is thus blocking anything after it. The smallest is deliberately not evicted to avoid a nonce-gap.
Analogously, if the pool is full, the consideration price of a new tx for evicting an old one is the smallest price in the entire nonce-sequence of the account. This avoids malicious users DoSing the pool with seemingly high paying transactions hidden behind a low-paying blocked one.
Since blob transactions have 3 price parameters: execution tip, execution fee cap and data fee cap, there's no singular parameter to create a total price ordering on. What's more, since the base fee and blob fee can move independently of one another, there's no pre-defined way to combine them into a stable order either. This leads to a multi-dimensional problem to solve after every block.
The first observation is that comparing 1559 base fees or 4844 blob fees needs to happen in the context of their dynamism. Since base fees are adjusted continuously and fluctuate, and we want to optimize for effective miner fees, it is better to disregard small base fee cap differences. Instead of considering the exact fee cap values, we should group transactions into buckets based on fee cap values, allowing us to use the miner tip meaningfully as a splitter inside a bucket.
To create these buckets, rather than looking at the absolute fee differences, the useful metric is the max time it can take to exceed the transaction's fee caps. Base fee changes are multiplicative, so we use a logarithmic scale. Fees jumps up or down in ~1.125 multipliers at max across blocks, so we use log1.125(fee) and rounding to eliminate noise. Specifically, we're interested in the number of jumps needed to go from the current fee to the transaction's cap:
jumps = floor(log1.125(txfee) - log1.125(basefee))
For blob fees, EIP-7892 changed the ratio of target to max blobs, and with that also the maximum blob fee decrease in a slot from 1.125 to approx 1.17. therefore, we use:
blobfeeJumps = floor(log1.17(txBlobfee) - log1.17(blobfee))
The second observation is that when ranking executable blob txs, it does not make sense to grant a later eviction priority to txs with high fee caps since it could enable pool wars. As such, any positive priority will be grouped together.
priority = min(jumps, 0)
The third observation is that the basefee and blobfee move independently, so there's no way to split mixed txs on their own (A has higher base fee, B has higher blob fee).
To establish a total order, we need to reduce the dimensionality of the two base fees (log jumps) to a single value. The interesting aspect from the pool's perspective is how fast will a tx get executable (fees going down, crossing the smaller negative jump counter) or non-executable (fees going up, crossing the smaller positive jump counter). As such, the pool cares only about the min of the two delta values for eviction priority.
priority = min(deltaBasefee, deltaBlobfee, 0)
The above very aggressive dimensionality and noise reduction should result in transaction being grouped into a small number of buckets, the further the fees the larger the buckets. This is good because it allows us to use the miner tip meaningfully as a splitter.
Optimisation tradeoffs:
- Eviction relies on 3 fee minimums per account (exec tip, exec cap and blob cap). Maintaining these values across all transactions from the account is problematic as each transaction replacement or inclusion would require a rescan of all other transactions to recalculate the minimum. Instead, the pool maintains a rolling minimum across the nonce range. Updating all the minimums will need to be done only starting at the swapped in/out nonce and leading up to the first no-change.
func New ¶
New creates a new blob transaction pool to gather, sort and filter inbound blob transactions from the network.
func (*BlobPool) Add ¶
func (p *BlobPool) Add(txs []*types.Transaction, sync bool) []error
Add inserts a set of blob transactions into the pool if they pass validation (both consensus validity and pool restrictions).
func (*BlobPool) AddPooledTx ¶ added in v1.17.5
func (p *BlobPool) AddPooledTx(ptx *BlobTxForPool) (err error)
AddPooledTx inserts a new blob transaction into the pool if it passes validation (both consensus validity and pool restrictions).
func (*BlobPool) Clear ¶ added in v1.14.12
func (p *BlobPool) Clear()
Clear implements txpool.SubPool, removing all tracked transactions from the blob pool and persistent store.
Note, do not use this in production / live code. In live code, the pool is meant to reset on a separate thread to avoid DoS vectors.
func (*BlobPool) Content ¶
func (p *BlobPool) Content() (map[common.Address][]*types.Transaction, map[common.Address][]*types.Transaction)
Content retrieves the data content of the transaction pool, returning all the pending as well as queued transactions, grouped by account and sorted by nonce.
For the blob pool, this method will return nothing for now. TODO(karalabe): Abstract out the returned metadata.
func (*BlobPool) ContentFrom ¶
func (p *BlobPool) ContentFrom(addr common.Address) ([]*types.Transaction, []*types.Transaction)
ContentFrom retrieves the data content of the transaction pool, returning the pending as well as queued transactions of this address, grouped by nonce.
For the blob pool, this method will return nothing for now. TODO(karalabe): Abstract out the returned metadata.
func (*BlobPool) Filter ¶
func (p *BlobPool) Filter(tx *types.Transaction) bool
Filter returns whether the given transaction can be consumed by the blob pool.
func (*BlobPool) FilterType ¶ added in v1.17.0
FilterType returns whether the blob pool supports the given transaction type.
func (*BlobPool) Get ¶
func (p *BlobPool) Get(hash common.Hash) *types.Transaction
Get returns a transaction if it is contained in the pool, or nil otherwise. Note that this function always try to recover full blobs
func (*BlobPool) GetBlobCells ¶ added in v1.17.5
func (p *BlobPool) GetBlobCells(vhashes []common.Hash, mask types.CustodyBitmap) ([][]*kzg4844.Cell, [][]*kzg4844.Proof, error)
GetBlobCells returns cells for the given versioned blob hashes. Nil entries mean that the cell was not available.
func (*BlobPool) GetBlobHashes ¶ added in v1.17.5
GetBlobHashes returns the blob versioned hashes for a given transaction hash.
func (*BlobPool) GetCustody ¶ added in v1.17.5
func (p *BlobPool) GetCustody(hash common.Hash) *types.CustodyBitmap
GetCustody returns the custody bitmap for a given transaction hash.
func (*BlobPool) GetMetadata ¶ added in v1.15.8
func (p *BlobPool) GetMetadata(hash common.Hash) *txpool.TxMetadata
GetMetadata returns the transaction type and transaction size with the given transaction hash.
The size refers the length of the 'rlp encoding' of a blob transaction including the attached blobs.
func (*BlobPool) GetRLP ¶ added in v1.15.6
GetRLP returns an RLP-encoded transaction if it is contained in the pool.
The encoded response is memoized: legacy (pre eth/72) peers fetch full blobs via GetPooledTransactions, and the same transaction is typically requested by multiple peers. Without the cache, each request re-reads the tx from disk and re-encodes it, recovering the blobs from the stored cells.
func (*BlobPool) Has ¶
Has returns an indicator whether subpool has a transaction cached with the given hash.
func (*BlobPool) Init ¶
Init sets the gas price needed to keep a transaction in the pool and the chain head to allow balance / nonce checks. The transaction journal will be loaded from disk and filtered based on the provided starting settings.
func (*BlobPool) Nonce ¶
Nonce returns the next nonce of an account, with all transactions executable by the pool already applied on top.
func (*BlobPool) Pending ¶
func (p *BlobPool) Pending(filter txpool.PendingFilter) (map[common.Address][]*txpool.LazyTransaction, int)
Pending retrieves all currently processable transactions, grouped by origin account and sorted by nonce.
The transactions can also be pre-filtered by the dynamic fee components to reduce allocations and load on downstream subsystems.
func (*BlobPool) Reset ¶
Reset implements txpool.SubPool, allowing the blob pool's internal state to be kept in sync with the main transaction pool's internal state.
func (*BlobPool) SetGasTip ¶
SetGasTip implements txpool.SubPool, allowing the blob pool's gas requirements to be kept in sync with the main transaction pool's gas requirements.
func (*BlobPool) Stats ¶
Stats retrieves the current pool stats, namely the number of pending and the number of queued (non-executable) transactions.
func (*BlobPool) Status ¶
Status returns the known status (unknown/pending/queued) of a transaction identified by their hashes.
func (*BlobPool) SubscribeTransactions ¶
func (p *BlobPool) SubscribeTransactions(ch chan<- core.NewTxsEvent, reorgs bool) event.Subscription
SubscribeTransactions registers a subscription for new transaction events, supporting feeding only newly seen or also resurrected transactions.
func (*BlobPool) ValidateTxBasics ¶ added in v1.15.4
func (p *BlobPool) ValidateTxBasics(tx *types.Transaction) error
ValidateTxBasics checks whether a transaction is valid according to the consensus rules, but does not check state-dependent validation such as sufficient balance. This check is meant as an early check which only needs to be performed once, and does not require the pool mutex to be held.
type BlobTxForPool ¶ added in v1.17.5
type BlobTxForPool struct {
Tx *types.Transaction // tx without sidecar
CellSidecar *types.BlobTxCellSidecar
}
BlobTxForPool is the storage representation of a blob transaction in the blobpool.
type BlockChain ¶
type BlockChain interface {
// Config retrieves the chain's fork configuration.
Config() *params.ChainConfig
// CurrentBlock returns the current head of the chain.
CurrentBlock() *types.Header
// CurrentFinalBlock returns the current block below which blobs should not
// be maintained anymore for reorg purposes.
CurrentFinalBlock() *types.Header
// GetBlock retrieves a specific block, used during pool resets.
GetBlock(hash common.Hash, number uint64) *types.Block
// StateAt returns a state database for a given chain header (generally the head).
StateAt(root common.Hash, number *big.Int, time uint64) (*state.StateDB, error)
}
BlockChain defines the minimal set of methods needed to back a blob pool with a chain. Exists to allow mocking the live chain out of tests.
type Cache ¶ added in v1.17.4
type Cache struct {
// contains filtered or unexported fields
}
Cache holds the blobs that are likely to be requested by the GetBlobs engine API.
Every `topKTimeout`, the cache selects the blobs of the top K most profitable transactions, and preloads them into the cache.
For HasBlobs requests, it also causes the blobs requested by the CL to be loaded. (Note: the cache is not guaranteed to always hold such blobs, since the blobpool might drop the transaction in the window between the engine API response and the cache update.)
func (*Cache) GetBlobs ¶ added in v1.17.4
func (c *Cache) GetBlobs(ctx context.Context, vhashes []common.Hash, version byte) (_ []*kzg4844.Blob, _ []kzg4844.Commitment, _ [][]kzg4844.Proof, err error)
GetBlobs returns the blobs and proofs for the given versioned hashes, serving them from the cache when possible and falling back to the blobpool for misses. Responses are placed in the order given in the request, using null for any missing blob.
For instance, if the request is [A_versioned_hash, B_versioned_hash, C_versioned_hash] and blobpool has data for blobs A and C, but doesn't have data for B, the response MUST be [A, null, C].
This is a utility method for the engine API, enabling consensus clients to retrieve blobs from the pools directly instead of the network.
The version argument specifies the type of proofs to return, either the blob proofs (version 0) or the cell proofs (version 1). Proofs conversion is CPU intensive and prohibited explicitly.
func (*Cache) GetCells ¶ added in v1.17.5
func (c *Cache) GetCells(vhashes []common.Hash, mask types.CustodyBitmap) ([][]*kzg4844.Cell, [][]*kzg4844.Proof, error)
GetCells returns cells for the given versioned blob hashes, filtered by the requested cell indices (mask). Each entry in the result corresponds to one vhash. Nil entries mean the cell was not available. If the cell is not available in cache, it falls back to the blobpool.
func (*Cache) HasBlobs ¶ added in v1.17.4
HasBlobs reports whether the blob is available (in the cache or the blobpool) and asks the loop to pin the ones it found.
func (*Cache) SetCellMode ¶ added in v1.17.5
EnableCell allows the cache to store only cells without recovering blobs. This means we can also cache cells that lack enough blobs to recover. It signals the loop to switch to cell mode and re-select transactions from this wider pool.
type Config ¶
type Config struct {
Datadir string // Data directory containing the currently executable blobs
Datacap uint64 // Soft-cap of database storage (hard cap is larger due to overhead)
PriceBump uint64 // Minimum price bump percentage to replace an already existing nonce
FetchProbability uint64 // EIP-8070: full blob fetch probability for sparse blobpool
BlockedRatio float64 // Maximum fraction of Datacap the blocked (partial-gated) suffix may occupy
}
Config are the configuration parameters of the blob transaction pool.
type PeerDelivery ¶ added in v1.17.5
PeerDelivery holds cells delivered by a single peer, in blob-major order.