Documentation
¶
Overview ¶
Package shadowsim runs a study's node as one real process per simulated host under the Shadow discrete-event simulator, on the network the study's own generators draw.
The in-process instruments (gossipsim) put N nodes in one process over a simulated network. This package is the second substrate for the same cells: N processes on real sockets, TCP or QUIC, under Shadow's virtual clock, with per-host bandwidth (to the kilobit) and per-pair latency (the matrix's upper triangle; Shadow's links are symmetric) taken from the study's topology. A cell defined once -- an arm, a seed, a payload, a node count and the environment that selects them -- runs on either substrate and yields the same record shape, so the two can be paired.
The contract has three parts, one per file:
- Topology (topology.go): the study exports the graph, the per-node access links and the one-way latency matrix for a cell as JSON. tools/gen_shadow.py turns that into Shadow's network graph and host list. The simulated network is the study's network by construction.
- Schedule (schedule.go) and the node (host.go): a node process reads its index, transport and timetable from SHADOW_* variables, builds a libp2p host on the transport, dials the edges it owns, and acts at fixed instants on the simulated clock, which Shadow keeps consistent across hosts. Identities derive from the cell seed, so every node can address every peer without an exchange.
- Stat (stat.go): each node prints one SHADOWSTAT line, its completion time and counters plus counter snapshots after the publish. tools/extract.py folds the lines into one result record with the driver's quantile rule and denominators; tools/compare.py pairs records across substrates.
tools/shadowrun.sh runs Shadow under a stall watchdog (a process spinning in userspace freezes Shadow's clock), and tools/lane.sh runs a cells file through a study's cell script, skipping cells whose record is already ok. tools/README.md has the mechanics and the record schema.
Index ¶
- Constants
- func Addr(transport, ip string, port int) (ma.Multiaddr, error)
- func AddrInfo(seed uint64, transport, ip string, port int, i int) (peer.AddrInfo, error)
- func DialEdges(ctx context.Context, h host.Host, index int, edges [][2]int, ...) error
- func Identity(seed uint64, i int) (crypto.PrivKey, error)
- func IsNode() bool
- func NewHost(priv crypto.PrivKey, transport, ip string, port int, extra ...libp2p.Option) (host.Host, error)
- func NodeIP(prefix string, i int) string
- func PeerID(seed uint64, i int) (peer.ID, error)
- func QUICReuse(ip net.IP) libp2p.Option
- func SimSeconds(at time.Time) float64
- func SleepUntil(at time.Time)
- type Schedule
- type Snapshot
- type Stat
- type Topology
Constants ¶
const ( EnvNode = "SHADOW_NODE" // set to 1 when the process is a Shadow node EnvIndex = "SHADOW_INDEX" // this node's index, 0 publishes EnvTransport = "SHADOW_TRANSPORT" // quic or tcp EnvPublishAt = "SHADOW_PUBLISH_AT" // seconds since the Unix epoch on the simulated clock EnvHoldMs = "SHADOW_HOLD_MS" // how long after the publish the node keeps counting EnvDialMs = "SHADOW_DIAL_MS" // when after start the node dials its edges (default 1000) EnvSubscribeMs = "SHADOW_SUBSCRIBE_MS" // when after start the node subscribes (default 10000) EnvDialTimeoutMs = "SHADOW_DIAL_TIMEOUT_MS" // how long a node keeps retrying one edge (default 30000) EnvDialSlots = "SHADOW_DIAL_SLOTS" // handshakes one node keeps in flight (default 0 = all) EnvPort = "SHADOW_PORT" // listening port (default 9000) EnvIPPrefix = "SHADOW_IP_PREFIX" // two-octet prefix (default 10.1) EnvEdges = "SHADOW_EDGES" // a study may honour line|star for known-answer cells EnvWarmAt = "SHADOW_WARM_AT" // seconds since the Unix epoch on the simulated clock at which a node warms its links; unset = none )
The node's environment. Everything about the cell (arm, payload, links, seed) comes from the study's own variables; these only place this process in the simulation.
const DefaultIPPrefix = topology.DefaultIPPrefix
DefaultIPPrefix is the /16 the nodes use unless SHADOW_IP_PREFIX says otherwise.
const SimStart = 946684800
SimStart is where Shadow's simulated clock begins: 2000-01-01T00:00:00Z. Absolute instants handed to a node (SHADOW_PUBLISH_AT) are seconds since the Unix epoch on that clock, so the generator computes them as this plus the simulated offset.
const StatPrefix = "SHADOWSTAT "
StatPrefix marks a node's report line on stdout.
Variables ¶
This section is empty.
Functions ¶
func AddrInfo ¶
AddrInfo is what a node needs to dial node i: its id under the seed and its address.
func DialEdges ¶
func DialEdges(ctx context.Context, h host.Host, index int, edges [][2]int, peerOf func(j int) (peer.AddrInfo, error), timeout time.Duration, concurrency int) error
DialEdges dials every edge this node owns (the pair's first index dials), retrying while the far end is still coming up, until timeout. peerOf resolves an index to its address.
concurrency bounds the handshakes this node has in flight; zero or less keeps them all in flight at once. On a real stack the bound matters: every node dialling its whole edge set together puts n*degree/2 handshakes on one host, and past a few thousand the drops and retransmits feed each other, so the setup rate collapses instead of saturating. Under simulated time there is no such cost and the bound changes nothing.
func Identity ¶
Identity derives node i's key from the cell seed, so peer ids are computable everywhere.
func NewHost ¶
func NewHost(priv crypto.PrivKey, transport, ip string, port int, extra ...libp2p.Option) (host.Host, error)
NewHost is a node's libp2p host: one transport, no relay, no resource limits, no address discovery -- the shape of the in-process instruments' blank hosts, on real sockets. TCP runs noise and yamux; QUIC runs quic-go on a plain packet connection (QUICReuse). Extra options go last.
func QUICReuse ¶
QUICReuse hands quic-go a plain packet connection, the way a simulated endpoint does: without raw-syscall access quic-go takes its basic path -- reads and writes, no don't-fragment, ECN, packet-info or batched-receive socket calls -- which is both what the in-process instruments run and what Shadow provides. The source-address selector is fixed to the node's own address, so dials reuse the listening socket instead of consulting the routing table.
func SimSeconds ¶
SimSeconds is an instant as seconds of simulated time since Shadow's clock started.
func SleepUntil ¶
SleepUntil blocks until the instant, which advances Shadow's clock; it returns at once if the instant has passed.
Types ¶
type Schedule ¶
type Schedule struct {
Index, N int
Transport string
Port int
IPPrefix string
PublishAt time.Time
Hold time.Duration
DialAt time.Duration // offset from the process start
// DialTimeout bounds the retries for one edge. Connection setup is the real stack's
// throughput limit: a host establishes a few hundred endpoints per second, so a large
// mesh needs a deadline proportional to its edge count, not a fixed 30 s.
DialTimeout time.Duration
// DialSlots bounds one node's in-flight handshakes; 0 dials the whole edge set at once.
DialSlots int
SubscribeAt time.Duration // offset from the process start
WarmAt time.Time // zero: no link warm-up; otherwise before PublishAt
}
Schedule is one node's own timetable.
func ScheduleFromEnv ¶
ScheduleFromEnv reads the node's timetable for a cell of n nodes; defaultHold applies when SHADOW_HOLD_MS is unset.
type Snapshot ¶
type Snapshot struct {
T float64 `json:"t"`
Dups int `json:"dups"`
RxBytes int `json:"rx_bytes"`
TxBytes int `json:"tx_bytes"`
CtrlRx int `json:"ctrl_rx"`
CtrlTx int `json:"ctrl_tx"`
IhaveSent int `json:"ihave_sent"`
IwantSent int `json:"iwant_sent"`
IwantRecv int `json:"iwant_recv"`
IwantIdsRecv int `json:"iwant_ids_recv"`
IdwSent int `json:"idw_sent"`
IdwRecv int `json:"idw_recv"`
Drops int `json:"drops"`
}
Snapshot is the tracer's counters at one instant after the publish.
type Stat ¶
type Stat struct {
Index int `json:"index"`
Transport string `json:"transport"`
Arm string `json:"arm"`
ArmName string `json:"arm_name"`
ArmMsgs int `json:"arm_msgs"`
ArmNeed int `json:"arm_need"`
ArmTotalBytes int `json:"arm_total_bytes"`
// The rest of the wire form: the unit the payload was cut at (0 for a whole message) and the
// message id width the node computes (0 where ids do not apply). With arm_msgs and arm_need
// they are what the comparison pairs on: two cells that differ in any are not a pair.
UnitBytes int `json:"unit_bytes"`
IDBytes int `json:"id_bytes"`
Completed bool `json:"completed"`
CompS float64 `json:"comp_s"` // seconds after the publish instant
MeshAtPublish int `json:"mesh_at_publish"`
PeersAtPublish int `json:"peers_at_publish"`
MeshAtEnd int `json:"mesh_at_end"`
Dups int `json:"dups"`
Rejects int `json:"rejects"`
RxBytes int `json:"rx_bytes"`
TxBytes int `json:"tx_bytes"`
CtrlRx int `json:"ctrl_rx"`
CtrlTx int `json:"ctrl_tx"`
IhaveSent int `json:"ihave_sent"`
IhaveRecv int `json:"ihave_recv"`
IwantSent int `json:"iwant_sent"`
IwantRecv int `json:"iwant_recv"`
IwantIdsSent int `json:"iwant_ids_sent"`
IwantIdsRecv int `json:"iwant_ids_recv"`
IdwSent int `json:"idw_sent"`
IdwRecv int `json:"idw_recv"`
GraftSent int `json:"graft_sent"`
PruneSent int `json:"prune_sent"`
DroppedRPCs int `json:"dropped_rpcs"`
Undeliverable int `json:"undeliverable"`
Declined int64 `json:"declined"`
// Extra carries a variant's own counters (variant B's pushed, served, requested segments),
// summed over the nodes by the extractor under the same keys.
Extra map[string]int64 `json:"extra,omitempty"`
StartS float64 `json:"start_s"` // simulated seconds at process start
PublishS float64 `json:"publish_s"` // the publish instant
EndS float64 `json:"end_s"` // the end of the hold
// Snapshots of the counters at fixed offsets from the publish, so the extractor can cut the
// aggregate where the in-process driver cuts it: at the last node's completion. The totals
// above run to the end of the hold and include what arrives after every node is done.
Snapshots []Snapshot `json:"snapshots"`
// Per-segment diffusion, one entry per message in the arm, in publish-relative seconds,
// with -1 for a segment this node never saw. FirstGotS is when the segment was delivered,
// FirstHeardS when the node first learned it existed -- an IHAVE, an IWANT, an IDONTWANT
// naming its id, or the delivery itself, whichever came first. Together they separate the
// two questions a single completion time cannot: how one segment spread, and how far ahead
// of holding it a node knew of it. Absent unless the cell asked for them, because one array
// pair per node is a different order of record size.
FirstGotS []float64 `json:"first_got_s,omitempty"`
FirstHeardS []float64 `json:"first_heard_s,omitempty"`
}
Stat is one node's report: what it ran, whether and when it completed, and the tracer's counters at the end of the hold. Byte and control counters follow the gossipsim tracer's vocabulary, so a record folded from these pairs with the in-process instruments' summary.
type Topology ¶
Topology is the shared topology artifact. The alias preserves existing study source, method sets and JSON while allowing other backends to use it directly.
func ReadTopology ¶
ReadTopology loads and validates the legacy topology artifact.