shadowsim

package
v0.0.0-...-f108f3b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 22, 2026 License: MIT Imports: 27 Imported by: 0

Documentation

Overview

Package shadowsim runs a study's node as one real process per simulated host under the Shadow discrete-event simulator, on the network the study's own generators draw.

The in-process instruments (gossipsim) put N nodes in one process over a simulated network. This package is the second substrate for the same cells: N processes on real sockets, TCP or QUIC, under Shadow's virtual clock, with per-host bandwidth (to the kilobit) and per-pair latency (the matrix's upper triangle; Shadow's links are symmetric) taken from the study's topology. A cell defined once -- an arm, a seed, a payload, a node count and the environment that selects them -- runs on either substrate and yields the same record shape, so the two can be paired.

The contract has three parts, one per file:

  • Topology (topology.go): the study exports the graph, the per-node access links and the one-way latency matrix for a cell as JSON. tools/gen_shadow.py turns that into Shadow's network graph and host list. The simulated network is the study's network by construction.
  • Schedule (schedule.go) and the node (host.go): a node process reads its index, transport and timetable from SHADOW_* variables, builds a libp2p host on the transport, dials the edges it owns, and acts at fixed instants on the simulated clock, which Shadow keeps consistent across hosts. Identities derive from the cell seed, so every node can address every peer without an exchange.
  • Stat (stat.go): each node prints one SHADOWSTAT line, its completion time and counters plus counter snapshots after the publish. tools/extract.py folds the lines into one result record with the driver's quantile rule and denominators; tools/compare.py pairs records across substrates.

tools/shadowrun.sh runs Shadow under a stall watchdog (a process spinning in userspace freezes Shadow's clock), and tools/lane.sh runs a cells file through a study's cell script, skipping cells whose record is already ok. tools/README.md has the mechanics and the record schema.

Index

Constants

View Source
const (
	EnvNode          = "SHADOW_NODE"            // set to 1 when the process is a Shadow node
	EnvIndex         = "SHADOW_INDEX"           // this node's index, 0 publishes
	EnvTransport     = "SHADOW_TRANSPORT"       // quic or tcp
	EnvPublishAt     = "SHADOW_PUBLISH_AT"      // seconds since the Unix epoch on the simulated clock
	EnvHoldMs        = "SHADOW_HOLD_MS"         // how long after the publish the node keeps counting
	EnvDialMs        = "SHADOW_DIAL_MS"         // when after start the node dials its edges (default 1000)
	EnvSubscribeMs   = "SHADOW_SUBSCRIBE_MS"    // when after start the node subscribes (default 10000)
	EnvDialTimeoutMs = "SHADOW_DIAL_TIMEOUT_MS" // how long a node keeps retrying one edge (default 30000)
	EnvDialSlots     = "SHADOW_DIAL_SLOTS"      // handshakes one node keeps in flight (default 0 = all)
	EnvPort          = "SHADOW_PORT"            // listening port (default 9000)
	EnvIPPrefix      = "SHADOW_IP_PREFIX"       // two-octet prefix (default 10.1)
	EnvEdges         = "SHADOW_EDGES"           // a study may honour line|star for known-answer cells
	EnvWarmAt        = "SHADOW_WARM_AT"         // seconds since the Unix epoch on the simulated clock at which a node warms its links; unset = none
)

The node's environment. Everything about the cell (arm, payload, links, seed) comes from the study's own variables; these only place this process in the simulation.

View Source
const DefaultIPPrefix = topology.DefaultIPPrefix

DefaultIPPrefix is the /16 the nodes use unless SHADOW_IP_PREFIX says otherwise.

View Source
const SimStart = 946684800

SimStart is where Shadow's simulated clock begins: 2000-01-01T00:00:00Z. Absolute instants handed to a node (SHADOW_PUBLISH_AT) are seconds since the Unix epoch on that clock, so the generator computes them as this plus the simulated offset.

View Source
const StatPrefix = "SHADOWSTAT "

StatPrefix marks a node's report line on stdout.

Variables

This section is empty.

Functions

func Addr

func Addr(transport, ip string, port int) (ma.Multiaddr, error)

Addr is a node's listening address on the transport.

func AddrInfo

func AddrInfo(seed uint64, transport, ip string, port int, i int) (peer.AddrInfo, error)

AddrInfo is what a node needs to dial node i: its id under the seed and its address.

func DialEdges

func DialEdges(ctx context.Context, h host.Host, index int, edges [][2]int, peerOf func(j int) (peer.AddrInfo, error), timeout time.Duration, concurrency int) error

DialEdges dials every edge this node owns (the pair's first index dials), retrying while the far end is still coming up, until timeout. peerOf resolves an index to its address.

concurrency bounds the handshakes this node has in flight; zero or less keeps them all in flight at once. On a real stack the bound matters: every node dialling its whole edge set together puts n*degree/2 handshakes on one host, and past a few thousand the drops and retransmits feed each other, so the setup rate collapses instead of saturating. Under simulated time there is no such cost and the bound changes nothing.

func Identity

func Identity(seed uint64, i int) (crypto.PrivKey, error)

Identity derives node i's key from the cell seed, so peer ids are computable everywhere.

func IsNode

func IsNode() bool

IsNode reports whether this process runs as a Shadow node.

func NewHost

func NewHost(priv crypto.PrivKey, transport, ip string, port int, extra ...libp2p.Option) (host.Host, error)

NewHost is a node's libp2p host: one transport, no relay, no resource limits, no address discovery -- the shape of the in-process instruments' blank hosts, on real sockets. TCP runs noise and yamux; QUIC runs quic-go on a plain packet connection (QUICReuse). Extra options go last.

func NodeIP

func NodeIP(prefix string, i int) string

NodeIP retains the legacy node address plan under a two-octet prefix.

func PeerID

func PeerID(seed uint64, i int) (peer.ID, error)

PeerID is node i's peer id under the cell seed.

func QUICReuse

func QUICReuse(ip net.IP) libp2p.Option

QUICReuse hands quic-go a plain packet connection, the way a simulated endpoint does: without raw-syscall access quic-go takes its basic path -- reads and writes, no don't-fragment, ECN, packet-info or batched-receive socket calls -- which is both what the in-process instruments run and what Shadow provides. The source-address selector is fixed to the node's own address, so dials reuse the listening socket instead of consulting the routing table.

func SimSeconds

func SimSeconds(at time.Time) float64

SimSeconds is an instant as seconds of simulated time since Shadow's clock started.

func SleepUntil

func SleepUntil(at time.Time)

SleepUntil blocks until the instant, which advances Shadow's clock; it returns at once if the instant has passed.

Types

type Schedule

type Schedule struct {
	Index, N  int
	Transport string
	Port      int
	IPPrefix  string
	PublishAt time.Time
	Hold      time.Duration
	DialAt    time.Duration // offset from the process start
	// DialTimeout bounds the retries for one edge. Connection setup is the real stack's
	// throughput limit: a host establishes a few hundred endpoints per second, so a large
	// mesh needs a deadline proportional to its edge count, not a fixed 30 s.
	DialTimeout time.Duration
	// DialSlots bounds one node's in-flight handshakes; 0 dials the whole edge set at once.
	DialSlots   int
	SubscribeAt time.Duration // offset from the process start
	WarmAt      time.Time     // zero: no link warm-up; otherwise before PublishAt
}

Schedule is one node's own timetable.

func ScheduleFromEnv

func ScheduleFromEnv(n int, defaultHold time.Duration) (Schedule, error)

ScheduleFromEnv reads the node's timetable for a cell of n nodes; defaultHold applies when SHADOW_HOLD_MS is unset.

func (Schedule) IP

func (s Schedule) IP(i int) string

IP is node i's address on this schedule's prefix.

type Snapshot

type Snapshot struct {
	T            float64 `json:"t"`
	Dups         int     `json:"dups"`
	RxBytes      int     `json:"rx_bytes"`
	TxBytes      int     `json:"tx_bytes"`
	CtrlRx       int     `json:"ctrl_rx"`
	CtrlTx       int     `json:"ctrl_tx"`
	IhaveSent    int     `json:"ihave_sent"`
	IwantSent    int     `json:"iwant_sent"`
	IwantRecv    int     `json:"iwant_recv"`
	IwantIdsRecv int     `json:"iwant_ids_recv"`
	IdwSent      int     `json:"idw_sent"`
	IdwRecv      int     `json:"idw_recv"`
	Drops        int     `json:"drops"`
}

Snapshot is the tracer's counters at one instant after the publish.

type Stat

type Stat struct {
	Index         int    `json:"index"`
	Transport     string `json:"transport"`
	Arm           string `json:"arm"`
	ArmName       string `json:"arm_name"`
	ArmMsgs       int    `json:"arm_msgs"`
	ArmNeed       int    `json:"arm_need"`
	ArmTotalBytes int    `json:"arm_total_bytes"`
	// The rest of the wire form: the unit the payload was cut at (0 for a whole message) and the
	// message id width the node computes (0 where ids do not apply). With arm_msgs and arm_need
	// they are what the comparison pairs on: two cells that differ in any are not a pair.
	UnitBytes      int     `json:"unit_bytes"`
	IDBytes        int     `json:"id_bytes"`
	Completed      bool    `json:"completed"`
	CompS          float64 `json:"comp_s"` // seconds after the publish instant
	MeshAtPublish  int     `json:"mesh_at_publish"`
	PeersAtPublish int     `json:"peers_at_publish"`
	MeshAtEnd      int     `json:"mesh_at_end"`
	Dups           int     `json:"dups"`
	Rejects        int     `json:"rejects"`
	RxBytes        int     `json:"rx_bytes"`
	TxBytes        int     `json:"tx_bytes"`
	CtrlRx         int     `json:"ctrl_rx"`
	CtrlTx         int     `json:"ctrl_tx"`
	IhaveSent      int     `json:"ihave_sent"`
	IhaveRecv      int     `json:"ihave_recv"`
	IwantSent      int     `json:"iwant_sent"`
	IwantRecv      int     `json:"iwant_recv"`
	IwantIdsSent   int     `json:"iwant_ids_sent"`
	IwantIdsRecv   int     `json:"iwant_ids_recv"`
	IdwSent        int     `json:"idw_sent"`
	IdwRecv        int     `json:"idw_recv"`
	GraftSent      int     `json:"graft_sent"`
	PruneSent      int     `json:"prune_sent"`
	DroppedRPCs    int     `json:"dropped_rpcs"`
	Undeliverable  int     `json:"undeliverable"`
	Declined       int64   `json:"declined"`
	// Extra carries a variant's own counters (variant B's pushed, served, requested segments),
	// summed over the nodes by the extractor under the same keys.
	Extra    map[string]int64 `json:"extra,omitempty"`
	StartS   float64          `json:"start_s"`   // simulated seconds at process start
	PublishS float64          `json:"publish_s"` // the publish instant
	EndS     float64          `json:"end_s"`     // the end of the hold
	// Snapshots of the counters at fixed offsets from the publish, so the extractor can cut the
	// aggregate where the in-process driver cuts it: at the last node's completion. The totals
	// above run to the end of the hold and include what arrives after every node is done.
	Snapshots []Snapshot `json:"snapshots"`
	// Per-segment diffusion, one entry per message in the arm, in publish-relative seconds,
	// with -1 for a segment this node never saw. FirstGotS is when the segment was delivered,
	// FirstHeardS when the node first learned it existed -- an IHAVE, an IWANT, an IDONTWANT
	// naming its id, or the delivery itself, whichever came first. Together they separate the
	// two questions a single completion time cannot: how one segment spread, and how far ahead
	// of holding it a node knew of it. Absent unless the cell asked for them, because one array
	// pair per node is a different order of record size.
	FirstGotS   []float64 `json:"first_got_s,omitempty"`
	FirstHeardS []float64 `json:"first_heard_s,omitempty"`
}

Stat is one node's report: what it ran, whether and when it completed, and the tracer's counters at the end of the hold. Byte and control counters follow the gossipsim tracer's vocabulary, so a record folded from these pairs with the in-process instruments' summary.

func ParseStat

func ParseStat(line string) (s Stat, ok bool, err error)

ParseStat reads a report line; ok is false for any other line.

func (Stat) Write

func (s Stat) Write(w io.Writer) error

Write prints the report as one prefixed JSON line.

type Topology

type Topology = topology.Topology

Topology is the shared topology artifact. The alias preserves existing study source, method sets and JSON while allowing other backends to use it directly.

func ReadTopology

func ReadTopology(path string) (*Topology, error)

ReadTopology loads and validates the legacy topology artifact.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址