|
| 1 | +# Issue 981 coverage closeout |
| 2 | + |
| 3 | +This suite closes the remaining benchmark wiring gaps in |
| 4 | +[#981](https://github.com/octanejs/octane/issues/981). It measures deterministic |
| 5 | +production work and checks observable behavior. It does not measure latency, |
| 6 | +layout, paint, garbage collection, or actual V8 heap allocations. |
| 7 | + |
| 8 | +```sh |
| 9 | +node benchmarks/bench.mjs --quick --ratios audit-981-coverage |
| 10 | +BENCH_JSON=/tmp/naive.json node benchmarks/audit-981-coverage/naive.mjs |
| 11 | +BENCH_JSON=/tmp/suspense-list.json node benchmarks/audit-981-coverage/suspense-list.mjs |
| 12 | +``` |
| 13 | + |
| 14 | +Both scripts build their own fixtures, serve them on an ephemeral loopback port, |
| 15 | +launch installed Playwright Chromium, and clean up temporary builds and servers. |
| 16 | +They require no separately started dev server. The unified runner merges their |
| 17 | +distinct targets; the weekly/manual Bench workflow runs that manifest without a |
| 18 | +suite filter. |
| 19 | + |
| 20 | +## Actual naive JSX and TSRX work |
| 21 | + |
| 22 | +`naive.mjs` builds the existing `js-framework/octane-jsx-naive` and |
| 23 | +`octane-tsrx-naive` applications with their actual Vite compiler configurations. |
| 24 | +It invokes the existing `js-framework/style-work.mjs` driver for mount, |
| 25 | +selection, a second selection, and an unrelated label update over 1,000 rows. |
| 26 | +There is no replacement fixture or source counter inside the compiled bodies. |
| 27 | + |
| 28 | +The driver now supports `WORK_CLEAN=1`: the same production build and actions |
| 29 | +without Chromium precise coverage or CSS interception. Seeded labels make the |
| 30 | +complete row HTML comparable across separate realms. Both passes independently |
| 31 | +check every label, row identity, complete relevant CSS, selection and native |
| 32 | +click handling; clean and observed HTML hashes, row counts and selections must |
| 33 | +match. Browser errors fail the driver. Instrumented CSS methods forward to the |
| 34 | +original implementation and are restored before semantic checks. |
| 35 | + |
| 36 | +Chromium `--jitless` precise coverage observes named entries in the readable |
| 37 | +production asset; there are no source instrumentation changes. Missing named |
| 38 | +row/runtime coverage fails before guards run. This is the existing style-work |
| 39 | +counter contract, now reachable through the unified CLI and ratio system. |
| 40 | + |
| 41 | +| Operation | JSX `setStyle` | TSRX `setStyle` | JSX `renderBlockInner` | TSRX `renderBlockInner` | Native style writes, both | |
| 42 | +| --- | ---: | ---: | ---: | ---: | ---: | |
| 43 | +| Mount 1,000 | 1,000 | 0 | 3,002 | 2,001 | 1,000 | |
| 44 | +| Select one | 1,000 | 0 | 3,002 | 2,001 | 1 | |
| 45 | +| Select another | 1,000 | 0 | 3,002 | 2,001 | 2 | |
| 46 | +| Unrelated label update | 1,000 | 0 | 3,002 | 2,001 | 0 | |
| 47 | + |
| 48 | +All operations retain zero style removals. The adapter reports additional |
| 49 | +`applyStyleValue`, `applyStyleProperty`, `setStyleProperty`, row-body and named |
| 50 | +CSS-property counts from the original driver. The 32 new ratios cover the four |
| 51 | +columns of work represented by `setStyle`, `renderBlockInner`, `styleSets`, and |
| 52 | +`styleRemoves`; the existing driver retains its complete exact work assertions. |
| 53 | +Every reference target represents the positive 1,000 rendered-row denominator. |
| 54 | +These guards establish a baseline for this existing workload; this PR claims |
| 55 | +no reduction in its measured counts. |
| 56 | + |
| 57 | +## Client Suspense inside a keyed list |
| 58 | + |
| 59 | +`suspense-list.mjs` compiles 64 keyed rows with two distinct memo wrappers and an |
| 60 | +independent `@try` boundary inside every row. Native promises settle the odd rows, |
| 61 | +then the even rows. The workload then changes an unrelated parent heading, |
| 62 | +reverses the list, and unmounts it. |
| 63 | + |
| 64 | +The clean minified bundle and readable observed bundle come from the same |
| 65 | +compiled, tree-shaken production graph. Their full HTML and lifecycle snapshots |
| 66 | +must match after every phase. Checks include: |
| 67 | + |
| 68 | +- Exactly the appropriate visible resolved and pending rows. A hidden staged |
| 69 | + button does not count as a revealed boundary. |
| 70 | +- The resolved value belongs to its own keyed row, including out-of-order waves. |
| 71 | +- Every surviving row keeps its DOM node and input draft through settlement, |
| 72 | + equal updates and reorder; settlement also preserves the focused input. |
| 73 | +- Native button events read the current resolved value. |
| 74 | +- Each resolved effect mounts once, no survivor cleans up during updates, and |
| 75 | + unmount clears the root and cleans up every effect exactly once. |
| 76 | + |
| 77 | +Precise coverage starts after module initialization and the root/data setup. |
| 78 | +Each operation's coverage ends before semantic event dispatches and assertions, |
| 79 | +so those probes do not inflate the measured work. The compiler may give nested |
| 80 | +body functions numeric name suffixes; the collector folds only the two authored |
| 81 | +body-name families, within this fixture's single production asset. Every measured |
| 82 | +function must be positively reached in at least one phase, preventing a renamed |
| 83 | +or absent probe from silently satisfying zero-work ratios. |
| 84 | + |
| 85 | +| Phase | `createBlock` | `renderBlockInner` | `RowImpl` | `AsyncCellImpl` | `forBlock` | `reconcileKeyed` | |
| 86 | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | |
| 87 | +| Pending mount | 321 | 321 | 64 | 64 | 1 | 0 | |
| 88 | +| Settle odd rows | 0 | 64 | 0 | 32 | 0 | 0 | |
| 89 | +| Settle even rows | 0 | 64 | 0 | 32 | 0 | 0 | |
| 90 | +| Unrelated heading update | 0 | 1 | 0 | 0 | 0 | 0 | |
| 91 | +| Reverse settled rows | 0 | 1 | 0 | 0 | 1 | 1 | |
| 92 | +| Unmount | 0 | 0 | 0 | 0 | 0 | 0 | |
| 93 | + |
| 94 | +The 36 ratios bound these entries per positive 64-row reference. Zero |
| 95 | +`createBlock` calls means this helper was not reached; it is not an allocation |
| 96 | +profile. Counts include required boundary orchestration and preserve the |
| 97 | +unchanged/update controls rather than using a speculative reduced-work target. |
| 98 | + |
| 99 | +`CLIENT_SOURCE_ROOT=/absolute/frozen/checkout` selects the runtime source for the |
| 100 | +Suspense script while retaining this compiler, fixture, dependencies and options. |
| 101 | +The naive adapter uses the actual Vite fixture graph in its checkout; compare it |
| 102 | +by running the same adapter and enhanced driver in a separate frozen worktree. |
| 103 | + |
| 104 | +## Existing coverage that was already sufficient |
| 105 | + |
| 106 | +The original memo gap is stale: `memo-wall` already has the `Row` and `Inner` |
| 107 | +wrappers, two compiled/descriptor walls and nine ratios. The registered |
| 108 | +`memo-wrapper-shapes` suite covers 128 and 1,024 distinct client/server wrappers, |
| 109 | +live defaults and static-hoisting controls. Its V8 shape counts are diagnostics; |
| 110 | +they are deliberately not ordinary semantic assertions. |
| 111 | + |
| 112 | +The original SSR list gap is also stale: `streaming-ssr` places per-card `@try` |
| 113 | +boundaries under a keyed `@for`. The new client workload covers client Blocks, |
| 114 | +survivors and lifecycle separately. `descriptor-renderer` already has 163 |
| 115 | +deterministic guards; the naive application adapter adds a guard for the exact |
| 116 | +JSX authoring workload rather than relying on those narrower microbenchmarks. |
| 117 | + |
| 118 | +## Measurement provenance and limits |
| 119 | + |
| 120 | +The baseline is merged #1090, `248af4edc30c80498dec13dad4f6d7508f10f220`. |
| 121 | +Measurements use Node 24.20.0, the installed Chromium version recorded in JSON, |
| 122 | +macOS arm64, production compiler options and disabled HMR/profiling. The |
| 123 | +[measurement artifact](measurements.json) records source/build hashes and the |
| 124 | +baseline/final work. Counts were unchanged on the final local run. |
| 125 | + |
| 126 | +The new list is bounded coverage, not every Suspense/Activity/lazy combination |
| 127 | +or an assertion about hidden-class count. Existing branch, hydration, root hold, |
| 128 | +Activity, async-composition and streaming suites retain their broader contracts. |
| 129 | +There are no timing thresholds and no prior thresholds are weakened. A future |
| 130 | +change that intentionally changes named work should update its guard with fresh |
| 131 | +semantic evidence and a documented reason. |
0 commit comments