Skip to content

πŸ› Bug Report β€” Runtime APIs: new_module_registry reports ~4Γ— higher startup CPU for static imports (real cost or measurement phase?)Β #8442

Open

Description

@danieltroger

Enabling only new_module_registry raised our Worker's reported startup time. A minimal repro shows the same jump for statically imported modules. Modules behind unexecuted dynamic imports are unaffected.

Repro (https://gist.github.com/danieltroger/9e0e840495325a8c6f568e181bd4a766; no bindings or secrets): an entry with 20 generated modules of ~210 KiB each, imported either statically or lazily behind /load. wrangler check startup --dry-run, 5 fresh runs per case, median Active ms:

Wrangler (workerd) Imports Legacy New registry
4.134.0 (1.20260917.1) static 16.2 65.8
4.162.0 (1.20261113.1) static 16.2 66.7
both lazy ~0 ~0

Likely cause: in the CPU profile the new-registry time sits almost entirely in the profiler's own startup() wrapper (59-62 ms). The profiler starts after mf.ready and then imports the entry. With the legacy registry, module compilation seems to happen before the profiler starts; with the new one it happens inside the measured import. If so, this is an accounting difference, not a slowdown.

Production context: in our app, upload validation reported startup time 139 β†’ 446 ms with only this flag changed. We have not seen a matching cold-request latency increase.

Questions:

  1. Is the interpretation above right?
  2. Does the upload validator's startup_time_ms have the same phase difference?
  3. What is the right way to compare startup cost across the two registries? If this belongs in workers-sdk, we can move it there.

How this was made: an OpenAI Codex agent (GPT-6.1) built the repro and ran the measurements; Claude (Anthropic) wrote this summary; a human reviewed it before posting.

Activity

  1. onmax commented on Nov 16, 2026

    @onmax

    I measured the same question from outside the profiler: CPU time of the workerd process (utime + stime from /proc) from start until the first response, for an entry plus one 20 MB module of string literals. Median of 7 interleaved runs, ms:

    20 MB module legacy registry new_module_registry
    none (empty Worker) 110
    static import 220 250
    behind import(), never called 220 180
    as a Text module, never imported 220

    wrangler check startup on the same bundles, non-idle profile time over 3 runs: static import 0 to 9 ms with the legacy registry, 82 to 122 ms with the new one. import()-only: about 0 with both.

    So for question 1 it looks like mostly a measurement phase. The legacy registry does the same work before the profiler starts (220 vs 250 ms of process CPU), and the real increase here is about 30 ms. Two more things show up:

    • With the legacy registry, a module reached only through import() costs the same as a static one, so lazy imports don't cut startup at all. The new registry fixes that (220 β†’ 180).
    • Bytes still cost something without any compile: 180 vs 110 ms for an uncalled import(), and 220 for a Text module. check startup shows 0 for that part with either registry.

    Setup: miniflare 5.20261116.0-beta (workerd 1.20261116.1) for the CPU numbers, wrangler 4.163.0 for check startup, compatibility date 2026-11-01, no nodejs_compat, Node 24.21.0, Linux x64 on a loaded machine, so single runs vary by Β±40 ms.

    bench.mjs
    // Measure workerd CPU time up to readiness for a Worker whose large module is
    // only reachable through import(), with and without new_module_registry.
    // CPU time (utime + stime of the workerd process) is far less noisy than wall
    // time on a loaded machine.
    import { readFileSync, readdirSync } from "node:fs";
    import { Miniflare, convertV4MiniflareOptions } from "miniflare"; // 5.20261116.0-beta
    
    const MB = Number(process.env.LAZY_MB ?? 20);
    const parts = [];
    for (let i = 0; i < MB * 4; i++) parts.push(`export const s${i} = ${JSON.stringify(String(i).padEnd(250_000, "abcdefghij"))};`);
    const lazySource = parts.join("\n");
    const entry = `export default { async fetch(req) {
      if (new URL(req.url).pathname === "/lazy") { const m = await import("./lazy.mjs"); return new Response(String(m.s1.length)); }
      return new Response("ok");
    } }`;
    
    function options(variant) {
      const modules = [{ type: "ESModule", path: "/w/entry.mjs", contents: entry }];
      if (variant === "same-bytes-as-text") modules.push({ type: "Text", path: "/w/lazy.txt", contents: lazySource });
      else if (variant !== "none") modules.push({ type: "ESModule", path: "/w/lazy.mjs", contents: lazySource });
      if (variant.startsWith("static")) modules[0] = { type: "ESModule", path: "/w/entry.mjs", contents: 'import * as m from "./lazy.mjs";\nexport default { fetch: () => new Response(m.s1 ? "ok" : "no") }' };
      return convertV4MiniflareOptions({
        modules, modulesRoot: "/w",
        compatibilityDate: "2026-11-01",
        compatibilityFlags: variant.endsWith("new-registry") ? ["new_module_registry"] : [],
      });
    }
    
    const TICK_MS = 10; // getconf CLK_TCK = 100 on Linux
    function workerdCpuMs() {
      let total = 0;
      for (const task of readdirSync(`/proc/${process.pid}/task`)) {
        const kids = readFileSync(`/proc/${process.pid}/task/${task}/children`, "utf8").trim().split(/\s+/).filter(Boolean);
        for (const pid of kids) {
          const cmd = readFileSync(`/proc/${pid}/cmdline`, "utf8");
          if (!cmd.includes("workerd")) continue;
          const stat = readFileSync(`/proc/${pid}/stat`, "utf8").split(") ")[1].split(" ");
          total += (Number(stat[11]) + Number(stat[12])) * TICK_MS; // utime, stime
        }
      }
      return total;
    }
    
    async function once(variant) {
      const mf = new Miniflare(options(variant));
      await mf.ready;
      const body = await (await mf.dispatchFetch("http://x/")).text();
      const cpu = workerdCpuMs();
      await mf.dispose();
      if (body !== "ok") throw new Error(`unexpected body ${body}`);
      return cpu;
    }
    
    const variants = ["none", "same-bytes-as-text", "lazy-legacy", "lazy-new-registry", "static-legacy", "static-new-registry"];
    const runs = Number(process.argv[2] ?? 7);
    const results = Object.fromEntries(variants.map(v => [v, []]));
    console.log(`import()-only module: ${(lazySource.length / 1e6).toFixed(1)} MB of string literals`);
    await once("none");
    for (let r = 0; r < runs; r++) for (const v of variants) results[v].push(await once(v));
    const median = a => [...a].sort((x, y) => x - y)[Math.floor(a.length / 2)];
    for (const v of variants) console.log(v.padEnd(18), "median workerd CPU ms to first response", median(results[v]), "| runs", results[v].join(" "));
  2. danieltroger commented on Nov 17, 2026

    @danieltroger
    Author

    Follow-up with production data, which corrects one line of my report ("We have not seen a matching cold-request latency increase").

    We shipped new_module_registry as the only change on one production Worker (a server-rendered app with a large static import graph) on 16 Nov, read real traffic, and reverted it the same evening.

    First request on a new isolate, same route, same store, same data centre:

    without the flag with new_module_registry
    requests 195 71
    CPU time, p50 54 ms 227 ms
    wall time, p50 560 ms 757 ms

    Every first-request CPU value with the flag was above every value without it. Upload-validation startup moved the same way (150 ms to 270 ms). After the revert, first-request CPU p50 went back to 42-46 ms.

    So for this Worker the extra startup CPU is real on cold requests, not only an effect of where the profiler starts measuring. I can share the module graph shape or run a build you want tested.

  3. jasnell commented on Nov 17, 2026

    @jasnell
    Collaborator

    ok, good signal. We will investigate improvements!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      ⌂ 捒网址