Repository navigation
Release notes
Long-form per-release changelog for stream-chain.
The repo's README.md carries cliff-notes for users skimming GitHub; this
page is the canonical history with internal-change context, motivation, and
credits.
Patch release. One bug fix. No API changes.
-
readableFrom()andreadableWebStreamFrom()release their sources when a stream stops early. Abreakfromfor await,destroy(),cancel(), an abortedpipeTo(), a failing source, or astopcallsreturn()on each active iterator, innermost first, so a generator'sfinallyruns. Before this release, abreakskipped a source'sfinally. When the pump is waiting for demand, the iterators close one at a time, teardown waits for them, and a failing cleanup is reported. When a source operation is in progress,return()is called on all iterators at once and teardown doesn't wait, so a stalled source can't delay it (thx mayankdhingra02).
- The utils page describes the teardown of both adapters.
- The new Recipes page shows how to process a fetched response, parse Server-Sent Events, and feed a custom stream into a chain.
- The test suites of stream-join (158 tests) and stream-sorting (175 tests) pass against this release on Node, Bun, and Deno. Both packages build on these adapters.
Patch release. Two bug fixes. No API changes.
-
gen(),fun(),pipe(), and the/corechain()passnullandundefinedthrough again. Since 4.1.1 they dropped both at every stage, the rule of the stream drivers, while the documentation said they pass them on like any other value. They pass them again, as 4.1.0 did: between stages, as output, frommany(), from a generator, and from a flush. A stage that returns nothing now givesundefinedto the next stage, and afor awaitloop over the pipeline seesundefineditems. Return none to drop a value in every context. asStream(), asWebStream(), and the chain() fromstream-chainandstream-chain/webstill dropnullandundefined, since streams reserve them for end-of-stream. Regression tests intests/core/test-null.js,tests/node/test-chain-null.js, andtests/web/test-web-chain-null.js. -
A stream driver drops a
nullreturned through finalValue(). In a Nodechain()afinalValue(null)ended the stream and lost the values after it; in a Webchain()it emitted thenull. Both now drop it like any othernull, andgen()still emits it.
- The gen(), fun(), asStream(), defs, chain(), and Concepts pages name every driver that passes
nullthrough and every driver that drops it. - stream-json's test suite (897 tests on Node, Bun, and Deno, plus its TypeScript tests and type checks) passes against this release, as it does against 4.4.0.
Minor release. drain() ends a pipe() run on its stop without an exception, and a Stop can come from another library. New API: isStop() and stopErrorSymbol. One bug fix.
-
isStop(error)andstopErrorSymbol.isStop()checks the mark everyStopcarries, not its class, so a library can throw aStopof its own class, marked withSymbol.for('object-stream.stopError'), without depending on stream-chain. stream-chain's pipelines recognize a stop the same way. See isStop() and stopErrorSymbol.
-
drain()ends apipe()run on itsStopand returns the last value. Before,drain(pipe(...))rejected withStopwhen a stage stopped the pipeline, althoughpipe()had already flushed the stages after the stopper. This is a breaking change for code that caught thatStopto learn the run ended early: that code now takes the normal path. Iterate the run withfor awaitto see theStop. AStopfrom any other iterable, a gen() run included, still reaches the caller, since agen()pipeline keeps processing the inputs fed to it after a stop. See drain().
-
asyncBlockWriter()keeps a surrogate pair in one write. When a string it received ended with the first half of a surrogate pair and that string filled a block, the two halves went to separate writes, and each half was written asU+FFFD: a 4-byte character became 6 bytes. The writer now holds the first half for the next write. stringerToFile() and the file stringers built on the writer get the fix. Contributed by edsonsims.
- Every internal symbol is a well-known name too,
Symbol.for('object-stream.<name>'), as the public markers already were. - The chain() page now explains the
AbortErrorthatpipeline()fromstream/promisesreports after a stop when its last step is an async generator function. It is Node's behavior; aWritabledestination, the callback form, andfor awaitend cleanly. - stream-json's test suite (875 tests on Node, Bun, and Deno, plus its type checks) passes against this release, and
drain(pipe(parseFile(), …))with a stage that stops resolves, where 4.3.0 rejected withStop. - Dev dependencies bumped (
@types/node).
Minor release. Pipelines release their resources when they end early, a stop flushes the stages after it the same way everywhere, and a leading byte-order mark is dropped. New API: a flushable abort, close() on the file sinks, and an ignoreBOM option.
-
flushable(write, final, abort). The optional third function releases a stage's resources when the pipeline ends abnormally: an error in another stage, an earlybreak, a destroyed or cancelled stream, or astopby this stage or a later one. It is stored underabortSymbol, also available aschain.abortSymbol. gen(), pipe(), asStream(), and asWebStream() call it at most once, and never after the stage's ownfinalcompleted. See flushable(). -
close()on asyncBlockWriter() and stringerToFile(). It releases the file handle and drops the buffered tail. It is also the writer'sabort, so a pipeline closes the file when it ends early; call it yourself when you drive the writer some other way. -
ignoreBOM. fixUtf8Stream(), the JSONL parsers,parseFile(), andasyncBlockReader()takeignoreBOM: trueto keep a leading byte-order mark.
-
A
stopflushes the stages after the stopper, on every owner. Before,asStream()and the Node chain() flushed every stage after astop, the stages before the stopper included, whileasWebStream(), the webchain(), andpipe()flushed none, so a sink after the stopper (astringerToFile(), abatch()) lost what it buffered. Now the stages after the stopper are flushed, and the stopper and the stages before it are released through theirabort. On the Node owners, the stages before the stopper are no longer flushed. Withgen()andfun(), the flush you call withnonecovers only the stages after the stopper. See defs. -
A
Stoplands at the right stage. A thrownStopworks the same as a returnedstop, aStopthat leaves apipe()used as a stage stops at that stage, and a stop during the flush ends the stopper's output while the flush goes on with the stages after it. -
pipe()flushes before it releases. After astop, it flushes the stages after the stopper, then releases the stopper and the stages before it, as the stream owners do, so a failing release never costs the flushed output.
-
Teardown releases generator stages. When an
asStream()orasWebStream()stage is destroyed, cancelled, or aborted, a generator stage suspended at ayieldis closed, so itsfinallyruns; a generator whose values all get dropped is stopped before its next yield. Contributed by mkantautas. -
The Node
chain()destroys its inner streams when it is destroyed before both of its sides finish, asstream.pipeline()does, without passing the error down. -
A leading BOM is dropped.
fixUtf8Stream()andasyncBlockReader()drop a U+FEFF that starts the input, bytes or strings, on Node, Bun, Deno, and browsers, and keep a later one. stream-json's parsers, which start withfixUtf8Stream(), accept a BOM-prefixed document again once they take this release. Reported by nemphys. -
A
stopwith a failing cleanup surfaces the cleanup error alone, in place of anAggregateErrorthat held the internalStop.
- Each flushable stage gets its
finalor itsabort, once, never both: an owner records which stages finished, so no stage is aborted twice or after its ownfinalcompleted. - stream-json's test suite (762 tests on Node, Bun, and Deno, plus its type checks) passes against this release.
- Deno's
TextDecoderdrops a U+FEFF that starts a laterdecode()call in streaming mode (denoland/deno#37090), so the leading-BOM rule is applied by hand on every runtime. - Dev dependencies bumped (
@types/node,nano-benchmark,prettier).
Patch release. Three bug fixes. No API changes.
-
fun()keeps the outputs of overlapping calls apart. A composedfun()pipeline held one output buffer for all its calls, so calling it again before an asynchronous call settled mixed the two calls' values, and one of them could fail with aTypeErroror resolve tonull. Each call now collects into its own buffer, the way gen() always did. Reported by oss-security-shopify; the report was handled as a bug, not a security advisory, since both calls come from the same application. Regression tests intests/core/test-fun-direct.js. -
lines()handles a CRLF split across chunks. When\rended one chunk and\nstarted the next, the emitted line kept a trailing\r; the same text in one chunk did not. The JSONL parser was unaffected (JSON.parseignores the whitespace). First direct tests forlines()intests/core/test-lines.js. -
asyncBlockWriterrefuses every call after a failed write. After a failed block write, final write, or close, the next call used to reopen the file with'w', truncating what had been written, and carry on without an error. Every later call now throws anErrorwhosecauseis the original failure, and the file is kept as it was. New tests intests/node/test-asyncBlockWriter.js.
-
fixUtf8Stream()'s input contract is documented: one stream carries either bytes or strings, not both. Its unused internal buffer was removed, with no change in behavior. -
The JSONL splitter was audited for rescanning on every chunk and measured linear on Node and Bun, 1 to 16 MB in a single line;
bench/jsonl-long-line.jsstays as the regression meter. - Dev dependencies bumped (
@types/node,prettier,tape-six).
Patch release. Plugs a file-descriptor leak in asyncBlockWriter on its own write failure. No API changes.
-
asyncBlockWriterreleases itsFileHandlewhen its own block write fails. Previously only the final tail-write path (final()) released the handle on failure; a failure in a data-pass block write (or the lazy open it triggers) leaked the fd, because the flushable'sfinal()never runs on an aborted pipeline. The data-passwrite()is now guarded the same way: on failure it closes the handle before the error propagates, and a write+close double-fault still surfaces anAggregateError([writeErr, closeErr])in that order. This covers the stage's own I/O failures only — a resource-owning sink still cannot observe an unrelated stage throwing elsewhere in the pipe; release from your owncatch/finallyfor that case. Regression test intests/node/test-resource-cleanup.js.
Patch release. Fixes a regression introduced in 4.2.3: a failed pipe() could throw a spurious AggregateError carrying the same error twice. No API changes.
-
A failed
pipe()propagates the original error, not anAggregateError. 4.2.3 ranpipe()'s end-of-input flush in afinally, so on a data-pass error it re-drove the pipeline to flush. A stateful flushable that throws before consuming its buffer (e.g. stream-json'sjsonVerifieron malformed input) then re-ran the same work over the same dirty buffer and threw a second, logically-identical error — whichpipe()wrapped asAggregateError([E, E]), obscuring the real error and its{message, line, pos, offset}fields.pipe()now flushes only after the data pass completes: a throw propagates and the flush is skipped. An exception is not recoverable — a failed pipeline is not finalized.
-
The flush no longer runs on
stop, a throw, or an earlybreak. Finalizing a pipeline that didn't complete is wrong (it re-runs failed work and emits partial tails); the flush is reserved for clean completion. Resource-owning source stages still release inline (a generator source getsit.return()from the executor's abort path, unchanged since 4.2.2). A resource-owning sink that needs release on abort guards its own code withtry/catch, or the caller does so in acatch/finally— the exception always propagates intact, never suppressed, so it stays actionable. - The genuine cleanup-double-fault
AggregateErrorinasyncBlockReader/asyncBlockWriter/exec(an I/O op and a separateclose()both failing) is unchanged — those combine two genuinely distinct failures and never re-run the work that threw.
Patch release. Resource-cleanup fixes in the functional (pipe / gen) path and the asyncBlock* helpers, plus richer error reporting when cleanup itself fails. No API changes.
-
pipe()flushes in afinally. Astop, throw, or earlybreakin the data pass previously skipped the end-of-input flush, so a flushable sink'sfinal()(e.g.asyncBlockWriter/stringerToFileclosing itsFileHandle) never ran — a confirmed file-handle leak (measured +1 fd). The flush now always runs. -
asyncBlockWriter/asyncBlockReaderalways release their handle. A failed final write no longer skips the writer'sclose(); a throwingclose()in the reader no longer masks the body error.exec.abortno longer silently drops the source generator's cleanup error. -
Cleanup double-faults become an
AggregateError. When an operation and its cleanup both throw, the two are reported together —.errorsin chronological[first, second]order — instead of one masking the other. Wrapping happens only when the primary is a realError, sogen()'s non-Errorcancel sentinel keeps its identity. Single failures still propagate unwrapped (a bareStopis still aStop).
-
stop/finalValueare documented as supported from generators (none/manystay pointless there — a generator expresses both natively viayield/yield*). The rule: issuestop/finalValue, thenreturn. Stream pipelines absorb astopas a clean end; functional pipelines propagate it (catch for cleanup). - The
finalValueauto-exit-generator alternative was benchmarked and rejected (a consistent ~2–3% regression on the generator-stage hot path); the documented rule above is the contract.
Patch release. A correctness fix in the shared executor — source generators are now released when a pipeline ends abnormally, closing a file-handle leak on file-backed pipelines — plus removal of dead, undocumented runtime exports.
-
Source generators are released on abnormal termination.
exec's generator driver (nextGen) left a source generator suspended at itsyieldwhen iteration ended abnormally — a downstream stage throwing, a consumer cancelling (for-awaitbreak), or astop. Itsfinally {}never ran, so a resource-owning source leaked:asyncBlockReader'sFileHandlewas never closed, and Node's GC finalizer later raisedERR_INVALID_STATE: A FileHandle object was closed during garbage collection.nextGennow callsit.return()on abnormal exit (awaited for an async generator) before re-throwing the original error — matchingfor await...of'sAsyncIteratorClosesemantics, where the original error wins over any cleanup error. This affects the purepipe/drain(gen()) path used byparseFile()andstringerToFile()on errors or early termination; theasStream/asWebStreampaths already cleaned up via the NodeDuplex/ streamdestroylifecycle.
-
Dead, undocumented runtime exports. None were typed (all absent from the
.d.tssurface) or reachable through a documented entry point:- The legacy
gen.nextasync-generator trampoline —gen()has been a push→pull bridge over the shared executor since 4.1.0; the trampoline was retained "for compatibility" but had no callers. - The
fun.next/fun.collect/fun.asArrayattachments and their named exports —fun.jsnow exports{fun}only. Thecollect/asArrayfunctions remain (used internally). - The
exec.next/exec.flushproperty attachments on the internalexecfactory — the namednext/flushexports (consumed by the four compositors) are unchanged.
- The legacy
- Minor internal cleanup: one
catch {(optional catch binding) where the error binding was already unused.
Patch release. Factory-bundled JSONL entry points — the ergonomic adapter surface stream-json's JSONL needs to delegate here — plus a bugfix removing checkedParse(), which 4.2.0 exposed by mistake.
-
Factory-bundled JSONL entries under
stream-chain/node/jsonl/andstream-chain/web/jsonl/. Each is one factory that carries the substrate adapters as methods, sojsonlParser.asStream(options)/jsonlParser.asWebStream(options)(and the stringer equivalents) work without separately importing the suffixedparserStream/parserWebStreammodules.-
stream-chain/node/jsonl/parser.js—jsonlParser()returns thegen()chain;.asStream(NodeDuplex) and.asWebStream(Web pair) attached. -
stream-chain/web/jsonl/parser.js— browser-safe;.asWebStreamonly (never pulls innode:stream). -
stream-chain/node/jsonl/stringer.js—jsonlStringer()returns a NodeTransform;.asStreamis the factory itself,.asWebStreamreturns a WebTransformStream. -
stream-chain/web/jsonl/stringer.js— browser-safe; returns a WebTransformStream,.asWebStreamis itself. - Two curated barrels,
stream-chain/node/jsonlandstream-chain/web/jsonl, export{jsonlParser, jsonlStringer}for the substrate (newexportskeys back them). - The option/item type names match
stream-json's —JsonlParserOptions,JsonlItem,JsonlStringerOptions— and a no-opcheckErrors?option is accepted, sostream-json's deprecated JSONL users can migrate by changing only the import specifier. See jsonl § Factory-bundled entries for the full surface and a migration table.
-
- The four existing JSONL adapter modules now also export their option types (
ParserOptions,ParserWebStreamOptions,StringerOptions,StringerWebStreamOptions), previously declared but un-exported.
-
Removed the standalone
checkedParse()export fromstream-chain/jsonl/parser.js. It was exposed in 4.2.0 by mistake — it has no use case outside the parser module (used exactly once internally), and the bogusarguments.length-based public signature it shipped with was never exercised. It is inlined back into its single call site. The streaming parser, theerrorIndicatorAPI, andignoreErrorsare unaffected.
- The bundled entries delegate to the existing adapter modules; no parsing/serialization logic is duplicated. The Web-flavored entries are covered by the browser-safety test (no
node:*imports in their transitive graph). - Internal: the per-line JSONL parser was refactored to select its error-handling closure once up front (function-form vs constant
errorIndicator,ignoreErrors, default) rather than re-checking per line — behavior is unchanged. - These entries exist primarily to unblock
stream-json's JSONL delegation: with the adapter surface matched,stream-jsoncan repoint its JSONL re-exports here and eventually delete its own JSONL surface as an import-only swap.
Minor release. New JSONL file-edge components for local-file pipelines, a richer JSONL error-handling API absorbed from stream-json, and a /core chain string-source pass-through fix. Everything else is additive; the only borderline-breaking change is the /core chain no longer iterating a string input character-by-character.
-
JSONL file-edge composites under
stream-chain/jsonl/file/(Node-only).-
parseFile(options)— returns agen()pipeline shaped as(path) => AsyncGenerator<{key, value}>. Internally composesasyncBlockReaderwith the standard JSONL parser; forwards all parser options. -
stringerToFile(path, options)— returns agen()pipeline shaped as(value) => AsyncGenerator<never>. Terminal sink that composes the function-pipelinestringerwithasyncBlockWriter; the writer's flushablefinal()closes the file handle on flush. - Drop straight into
chain([...])(the chain factory wraps them into one fusedasStreaminternally), or drive directly withpipe(...)+drain(...)for the substrate-freegenpath. See jsonl § File-edge composites for the full story andbench/jsonl-file.jsfor the round-trip numbers.
-
-
JSONL
errorIndicatorAPI onparser({...}),parserStream, andparserWebStream. Presence-checked ('errorIndicator' in options), soerrorIndicator: undefinedis meaningful and distinct from "not set":-
errorIndicator: undefined— drop bad lines without bumping the line counter (sequential keys). -
errorIndicator: null(or any constant) — emit the value in place of the failed line, counter bumps. -
errorIndicator: (error, input, reviver) => unknown— function form; return value replaces the line,undefinedreturn drops without bumping. - The legacy
ignoreErrors: trueshortcut is preserved unchanged for backwards compatibility (drops but the counter bumps every line → gappy keys).errorIndicatorwins when both are set.
-
-
Raw
jsonlParser(options?)— per-line factory without thefixUtf8Stream -> linesinput front. For callers whose chunks already arrive line-aligned. Parallels stream-json'sjsonParserraw export. -
Standalone
checkedParse(input, reviver?, errorIndicator?)— single-line parser. Behaves exactly likeJSON.parse(input, reviver)unlesserrorIndicatoris passed, in which case parse failures invoke the function form (or return the constant) instead of throwing.arguments.length < 3is what distinguishes "omitted-indicator throws" from "explicitundefinedcatches". -
Function-pipeline JSONL
stringeratsrc/jsonl/stringer.js. Flushable; canonical building block alongside the existing Transform / TransformStream variants. Same options asstringerStream. -
Supporting helpers in
src/utils/:-
pipe(...stages)— one-shot single-value driver for agenpipeline. Runsg(value)theng(none)so flushable sinks close. Substrate-free. -
drain(asyncIter)— awaits an async iterable, returns its last yielded value. Standard way to await apipe(...)(value)whose output you don't otherwise want. -
asyncBlockReader(options?)— Node-only(path) => AsyncGenerator<string>overfs/promises.open+StringDecoder('utf8'). -
asyncBlockWriter(path, options?)— Node-only flushable that buffers and writes fixed-size blocks viafileHandle.write, closing the handle on flush.
-
-
/corechain string-source iteration footgun.chain([...])('input.json')previously walked'i','n','p',...into the first stage because strings carry aSymbol.iteratorand/core's outerfor await (const v of input) yield* g(v)happily iterated them. The driver now passes strings through as a single value, and the same check carves out other non-iterables (numbers, booleans, plain objects, …) so they don't throw onfor awaiteither. Arrays, generators, async iterables,Map,Setstill iterate as before;null/undefinedstill yield empty. Borderline-breaking — anyone relying on per-character iteration of a string source loses it (the explicit replacement is[...'hello']). Surfaced 2026-05-28 wiring stream-json'sparseFile. -
Empty JSONL lines now dropped silently across all parser variants. Was:
JSON.parse('')threw withoutignoreErrorsset. This is part of theerrorIndicatorwork and applies uniformly toparser/parserStream/parserWebStreamregardless of which error-handling option (if any) is set.
-
Bench.
bench/jsonl-file.js(50k JSONL rows fixture,nano-benchdriven) compares the new file-edge components against the equivalentfs.createReadStream + parserStream + ... + stringerStream + fs.createWriteStreamarrangement. Round-trip is ~40% faster; parse-with-in-pipeline-work is ~10% faster; parse-and-drain-externally is ~20% slower (the per-token gen async-bridge cost — same lesson as the sink-placement principle stream-json saw in its 2026-05-28 file-edge bench). See the bench header for which shape applies to which workload. -
stream-json proxy unblocked. The
errorIndicator/checkedParseAPI absorption makes stream-json'ssrc/core/jsonl/parser.jsa candidate to collapse to a proxy re-export of stream-chain's parser plus a couple of aliases. The stream-json side ships when its peerdep is bumped to^4.2.0(separate session). -
Helpers duplicated for now.
pipe,drain,asyncBlockReader, andasyncBlockWriterare intentionally mirrored between stream-chain and stream-json so each package stays self-contained. A future stream-json cleanup will collapse its copies into re-exports from stream-chain. -
Docs lead. README gained a new "JSONL" section with a
chain()-based round-trip example,wiki/jsonl.mdgrew a "File-edge composites" section,wiki/utils.mdgrew "Driving gen pipelines" and "Block I/O (Node-only)" sections,wiki/benchmarks.mdwas rewritten to match the actualbench/contents, andwiki/Home.mdindex links the new helpers and components.
Patch release. A performance release: the four compositors now share one
sync-when-possible executor, and a memory-growth bug on backpressured many()
expansion is fixed. No public API change and no behavior change — the full test
suite passes identically before and after.
-
Unified
execengine (src/exec.js).gen(),fun(),asStream(), andasWebStream()previously each ran their own fused-pipeline executor; the Node and Web wrappers used anasync function applyFnsthat returned a Promise on every call andawaited between every push. They now share a single sync-when-possible, value-or-promise executor: it threads a value through the function-list and emits via apushcallback, staying fully synchronous until the first real promise (an async stage, a thenable value, or a backpressuring push) appears — then it suspends and resumes. Synchronous pipelines no longer pay a per-item microtask.execis internal (not exported). -
gen()is now a push→pull bridge over the shared executor rather than a recursiveasync function*; the legacy trampoline is retained asgen.nextfor compatibility but is no longer used internally. - The bounded-queue guarantee is unchanged: the executor honors the
pushreturn value, so when an enqueue backpressures it suspends at that push and the readable queue stays athwm + 1regardless of how many values one input produces.
-
O(n)→O(1) allocation on backpressured
many(). When a single input expanded to a chunk-sizedmany([...])and the consumer backpressured from the first element, themany()(andflush) traversal allocated one closure + promise per remaining element — O(N) live allocations, observable as heavy GC on large fan-outs (e.g. a JSON tokenizer emitting onemany(tokens)per input buffer). Both paths are now resumable, allocating one closure per actual suspension. A heap-delta regression gate (bench/json-exec/correctness/bounded-alloc.js) pins the allocation flat across N = 1e4…1e6.
- Executor-layer microbenchmarks (synthetic, nano-bench):
asStream≈ 26% faster,asWebStream≈ 10% faster than the oldapplyFnspath;execis the fastest of the in-processfun/gen/exectrio. - End-to-end against stream-json via
npm link: realcreateReadStreamfile pipelines 20–26% faster (json-filter, json-transform, json-count); a chunk-size sweep that previously blew up on whole-document input is now flat across chunk sizes.
- No source change is required to benefit — the gains are automatic. See the updated Migration guide § 4 and asStream / asWebStream for the revised backpressure description (the earlier "~40% async penalty" caveat no longer applies).
Minor release. Web Streams parity across the utility surface. Every
Node-substrate primitive now has a Web counterpart with matching contract;
substrate-agnostic helpers (dataSource, stream type guards) are exposed
on every subpath.
-
readableWebStreamFrom(iterable | options)insrc/utils/readableWebStreamFrom.js. Web Streams counterpart toreadableFrom(). Accepts a plain iterable, async iterable, iterator, 0-ary function (sync or async), or{iterable, strategy?}options object. Returns aReadableStream. Per-item backpressure viadesiredSize/pull(). Speaks the chain protocol on the producer side (returnednone/null/undefinedis skipped,stopterminates,many([...])fans out,finalValue(v)is unwrapped). For the plain iterable case the platform'sReadableStream.from()is enough; reach forreadableWebStreamFromwhen you need function-source, promise-resolution, or chain-protocol awareness. -
reduceWebStream(reducer, initial) / reduceWebStream(options)insrc/utils/reduceWebStream.js. Web Streams counterpart toreduceStream(). Returns{writable, result, accumulator}— write intowritable,await resultfor the final accumulator (resolves on clean close, rejects onabortor reducer error), or read the running value any time via theaccumulatorgetter. Reducer is called withthisbound to the return object sothis.accumulatorworks inside (parity withreduceStream). -
parserWebStream(options?)insrc/jsonl/parserWebStream.js. Web Streams counterpart toparserStream. One-liner overasWebStream(parser({...})). Samereviver/ignoreErrorscontract. Emitted records are{key, value}wherekeyis the zero-based input line index. -
stringerWebStream(options?)insrc/jsonl/stringerWebStream.js. Web Streams counterpart tostringerStream. Implemented as aTransformStreambecauseJSON.stringifyis synchronous — the substrate'sflush()callback handles the suffix/emptyValueemit, and there's no value in routing terminal text-emission throughapplyFns's per-chunk Promise allocation (stringer sits downstream of user code, not between intermediate function stages). -
dataSource(fn)on/weband/core. Moved tosrc/dataSource.js(substrate-agnostic). Re-exported from all three subpath entries (stream-chain,stream-chain/web,stream-chain/core) and attached aschain.dataSourceon each. -
Node stream type guards in
defs.js.isReadableNodeStream,isWritableNodeStream,isDuplexNodeStreamjoined the Web guards. Shape-based (nonode:streamimport), so callable from any substrate without pulling Node Streams into a browser bundle. Closes the symmetry with the Web guards (which had been exported since 4.0.0).
-
parser({ignoreErrors: true})correctly drops bad lines silently. The wiki documented "all JSON parsing errors are ignored silently", but the wrapper was emitting{key, value: defs.none}envelopes for unparseable lines — leaking the internalnonesentinel to userland. Fix:src/jsonl/parser.jsnow returnsdefs.nonefrom the wrapper whenparseFnreturnsnone, lettingasStream/asWebStream's filter machinery drop the envelope. BothparserStreamandparserWebStreaminherit the fix through the sharedparser()core.
-
Modernization sweep:
options && options.X→options?.Xacross the source tree (20 sites, 10 files). For paired fallback chains, switched||to??only after per-site analysis confirmed the left-side type was object-or-undefined (QueuingStrategy, reviver-or-function). For initial-accumulator presence checks ('initial' in options), kept the explicitinoperator —??would coalesce explicitnullaccumulators to the default, losing the semantic distinction. -
reduceStream/reduceWebStreamcleanup. Dropped a band-aid'initial' in options && options.initial !== undefinedcheck that worked around a normalization clobber bug. The wrap step now only writesinitialinto options when it was actually passed positionally, and a clean'initial' in optionspresence check stands on its own. Both substrates aligned. -
Stream type guards consolidated in
defs.js. Node guards moved from inline definitions insrc/index.jsintodefs.jsalongside the Web guards.src/index.jsnow destructures them from thedefsnamespace like the Web guards. -
dataSourceextracted to its own module. Was inline insrc/index.js. Nowsrc/dataSource.js+.d.ts, parallel togen.js/fun.js. -
Wiki Node↔Web mapping table added to
wiki/utils.mdandwiki/jsonl.mdheaders.wiki/defs.mdgained a "Stream type guards" section covering both substrates.
Test count: 322 tests / ~620 asserts on Node (up from 282 / 562 in 4.0.2). Browser bucket: 176 tests (up from 137). New suites:
-
tests/web/test-readableWebStreamFrom.jsandtests/web/test-readableWebStreamFrom-lifecycle.js— smoke, protocol features, backpressure, cancel-halts-pump, error propagation. -
tests/web/test-reduceWebStream.js— positional + options forms, sync + async reducers, abort, reducer-throws,this.accumulatorNode-parity, liveaccumulatorgetter. -
tests/web/test-jsonl-parserWebStream.js— smoke, chunked multi-byte boundaries, bad-JSON propagation,ignoreErrorsdrops bad lines, reviver, roundtrip via stringerWebStream. -
tests/web/test-jsonl-stringerWebStream.jsandtests/web/test-jsonl-stringerWebStream-lifecycle.js— smoke, custom-separator edges, abort, cancel, replacer-throws, empty-stream-flush. -
tests/core/test-dataSource.js— substrate-agnostic dataSource behavior, per-subpath re-export. -
tests/node/test-jsonl-parserStream.js— addedignoreErrorstest that pins the silently-drops-bad-lines contract on the Node side.
All four runtimes green: Node 26 / Bun 1.3.14 / Deno 2.7 / headless
Chromium via tape-six-playwright. ts-check + js-check + Prettier
clean.
Patch release. Restores fixUtf8Stream() portability across runtimes
that don't expose node:string_decoder.
-
fixUtf8Stream()no longer breaks browser consumers. 4.0.1'simport {StringDecoder} from 'node:string_decoder'at module top crashed every browser ESM consumer of a fixUtf8Stream user on load — before any test or code ran. The barenode:specifier fails resolution in browsers, with no try/catch surface above.Fix:
src/utils/fixUtf8Stream.jsnow defaults to aTextDecoder-backed factory (WHATWG standard, available in every supported runtime) and, on Node specifically, asynchronously upgrades toStringDecoderviaimport('node:string_decoder').then(...). Bun and Deno stay onTextDecoderper benchmark (Bun: roughly tied; Deno'sTextDecoderactually beats its node-compatStringDecoder). Runtime detection uses the same idiom astape-six'sutils/timer.js.No top-level
await—tests/node/test-cjs.cjs(require-of-ESM interop) stays green; an earlier TLA shape broke it across Node, Bun, and Deno during exploration.Decision data:
bench/decoder-stringdecoder-vs-textdecoder.js(kept in-repo). Node 26:StringDecoder2–4× faster on the decoder hot path; Bun 1.3.14: wash; Deno 2.7:TextDecoderbeatsStringDecodervia node-compat.
-
whenReady()named export. Returns aPromise<void>that resolves when the Node-side upgrade has landed (or immediately on non-Node). Optional —fixUtf8Stream()always works; the dispatcher reads the current implementation at call time, so streams composed before the upgrade get theTextDecoderstage (correct, small overhead) and streams composed after get theStringDecoderstage.await whenReady()before composition guarantees the Node fast path from the first call.
src/utils/fixUtf8Stream.d.ts now types the chunk parameter as
string | Uint8Array | typeof none (was string | Buffer | typeof none).
On Node, Buffer extends Uint8Array, so existing Node consumers still
type-check. The /// <reference types="node" /> directive is gone — the
file no longer depends on Node-only types.
Patch release. Two cross-runtime compatibility fixes surfaced during a multi-runtime verification pass (Node 26, Bun 1.3.14, Deno 2.7, headless Chromium via tape-six-playwright).
-
asWebStream()crash on Bun ≤1.3.14.src/asWebStream.jslistens tocontroller.signalinside theWritableStreamsink'sstart(controller)callback to wake a backpressured write when the writer aborts — without it, an aborted writer with a pending drain would deadlock. The signal is per WHATWG Streams §4.5.2 and works on Node 26 and Deno 2.7, but Bun 1.3.14'sWritableStreamDefaultControllerreturnsundefinedfor.signal, causing everyasWebStreaminstantiation on Bun to throwTypeError: undefined is not an object (evaluating 'controller.signal.addEventListener').Fix: optional-chained the listener (
controller.signal?.addEventListener(...)). Bun loses the abort-wakeup safety net until upstream lands the fix — a consumer that aborts a writer with a pending backpressured write would deadlock, rare in practice. Normal write/close/cancel paths are unaffected.Upstream fix in flight: oven-sh/bun#31156 / PR #31157. The PR cites stream-chain v4 by name as the motivating downstream case and includes a regression test labeled "unblocks a pending
write()from an abort listener — the stream-chain v4 use case." When the PR ships in Bun 1.3.15+, the optional chaining becomes a no-op and the abort-wakeup safety net is restored across all four runtimes. -
fixUtf8Stream()barestring_decoderimport.src/utils/fixUtf8Stream.jsdidimport {StringDecoder} from 'string_decoder'. Node and Bun resolve this to the built-in fine, but Deno requires the explicitnode:prefix for any built-in module import (TypeError: Import "string_decoder" not a dependency). Fix:import {StringDecoder} from 'node:string_decoder'. ESM hygiene anyway — the bare specifier has been deprecated in favor ofnode:prefixes since Node 16.
The test directory is now organized by execution environment, mirroring
tape-six's configuration schema:
-
tests/core/— substrate-agnostic tests (use therunChain(transducers, input)helper fromtests/web-helpers.js, which drives a/webchain internally). Runs in browser AND CLI. -
tests/web/— Web Streams substrate. Runs in browser AND CLI. -
tests/node/— Node Streams substrate. Runs only in CLI.
Configured via tape6.tests (core + web) and tape6.cli (node) in
package.json. The split lets the same test set drive Node 26, Bun 1.3.14,
Deno 2.7, and headless Chromium without per-runtime patterns in the npm
scripts.
npm run test:browser drives headless Chromium through
tape-six-playwright,
auto-starting tape6-server on port 55555 (env-overridable to avoid the
default 3000 collision). Runs the browser-safe subset: 113 tests / 166
asserts covering asWebStream, /web chain, webStreamPuller, and defs
in a real browser engine.
Two Bun-specific feature-probe skips were added in this release —
tests/web/test-asWebStream-lifecycle.js and tests/web/test-webStreamPuller.js
each detect the relevant Bun spec gaps at test start and t.skipTest(...)
rather than failing. They auto-unskip when Bun ships the matching fixes,
reported as skipped: 2 in npm run test:bun output for visibility.
All four runtimes green:
| Runtime | tests / asserts | Status |
|---|---|---|
| Node 26 | 258 / 534 | all pass |
| Bun 1.3.14 | 258 / 532 | all pass (2 skipped via feature-probe) |
| Deno 2.7 | 258 / 534 | all pass |
| Browser (Chromium) | 113 / 166 | all pass |
Also covers Ubuntu 26.04: Playwright's manifest stops at ubuntu24.04, so
the npm install postinstall fails to download Chromium on fresh 26.04.
Workaround documented in AGENTS.md: npm install --ignore-scripts then
PLAYWRIGHT_HOST_PLATFORM_OVERRIDE=ubuntu24.04-x64 npx playwright install chromium. Install-time only; runtime needs no env.
Major release. Substantial substrate and API expansion; small but breaking adjustments for CJS callers and existing consumers. See the dedicated Migration guide for a step-by-step upgrade walkthrough.
-
ESM-only distribution. The package is
"type": "module". The 3.x fallback that exposedchainas a callable fromrequire('stream-chain')is gone — CJS callers now destructure:const {chain} = require('stream-chain'). Was the only required source change for most callers. - Node 22+ floor. Drops Node 16, 18, 20. Current supported majors are 22, 24, 26 (latest minor of each).
-
Per-item backpressure semantic shift. When a function returns
many(...)or a generator yields multiple values, the runtime now awaits the drain signal between every push. Queue stays athwm + 1regardless of expansion factor. Cost: ~40% throughput hit on synchronous-heavy pipelines (theapplyFnspath is nowasyncend-to-end). Benefit: bounded memory under unbounded expansion. No source change needed. -
Generators-yield-plain convention. Generators (sync
function*and asyncasync function*) must yield plain values only — notnone,stop,many(...), orfinalValue(...). 3.x was inconsistent about handling these from generator output; 4.x is explicit that the behavior is undefined.
The default stream-chain entry resolves to stream-chain/node (the
canonical Node Streams runtime). Two new substrate-specific subpaths:
-
stream-chain/web— a native Web Streams chain. Returns{readable, writable}pairs built onReadableStream/WritableStream. Nonode:streaminterop layer — browser-safe and works in any environment with the Web Streams substrate. -
stream-chain/core— substrate-free composition. Returns a callable async-iterable factory:(input?) => AsyncGenerator<R>. Useful forgen()/fun()composition without dragging in either substrate.
Existing 3.x code keeps working without source changes — the split is purely additive.
Web Streams counterpart to asStream(). Returns a native
{readable, writable} duplex pair (NOT a TransformStream — its
transform() callback can't suspend mid-call for per-item drain).
Per-item backpressure via a custom pull() callback that wakes a
pending-drain Promise. Options accept Web Streams' standard
QueuingStrategy shape: {strategy?, readableStrategy?, writableStrategy?}.
Lifecycle matches TransformStream: reader.cancel(reason) propagates
to the writable as an error; writer.abort(reason) errors both sides
and unblocks pending backpressure; user-function errors propagate to
both sides. See asWebStream().
isReadableWebStream, isWritableWebStream, isDuplexWebStream
consolidated into stream-chain/defs.js as the single canonical source.
Re-exported from stream-chain (and chain.X) and stream-chain/web.
makeStreamPuller(readable) wraps a Node Readable as a non-destructive
async iterator (thin facade over readable.iterator({destroyOnReturn: false})). Preserves the original 'error' value (no AbortError
wrapping), synthesizes Error('Premature close') on destroy-without-end,
and breaks out of for await without destroying the source.
makeWebStreamPuller(readable) wraps a Web ReadableStream similarly
(built on stream[Symbol.asyncIterator]({preventCancel: true})) plus a
cancel(reason) extension method — the iterator-protocol return()
can't carry a cancel reason cleanly.
Both intended for downstream consumers (stream-join, stream-sorting)
that need original-error preservation. Both implement [Symbol.asyncIterator]
directly so for await works without ceremony. Live at
stream-chain/utils/streamPuller.js and stream-chain/utils/webStreamPuller.js.
See makeStreamPuller() and makeWebStreamPuller().
Five distinct issues found and fixed during the post-merge cleanup pass.
Each ships with regression tests in tests/test-asStream-lifecycle.js and
tests/test-asWebStream-lifecycle.js (257 / 534 asserts total, up from
237 / 513 in 3.6.3):
-
asWebStream
writer.abort()deadlock when a sink write was awaiting backpressure. Spec serializes sink callbacks, so theabortsink couldn't fire to unblock the pending drain. Fix: listen tocontroller.signalinstart()(the signal aborts before sink callbacks settle). -
asWebStream
reader.cancel()left writes silently failing. Should match TransformStream: propagate the cancel reason to the writable as an error so the producer learns. As a side effect, mid-chain errors in/webchains now reach the producer. -
asWebStream double-close on the readable controller after cancel.
Idempotent
closeReadable/errorReadablehelpers introduced. -
asWebStream user-function errors weren't propagating to the readable.
Downstream
pipeTo'd stages would hang waiting for input. Fix:errorReadable(error)inwrite/closesink catch blocks before rethrowing. -
asStream
stream.destroy()deadlock while in-flight_writewas awaitingpaused. Node doesn't signal in-flight writes either. Fix: customdestroy(err, callback)sink in the Duplex options that callsresume().
-
isNodeStreamdefensive helper insrc/index.jsremoved — no longer needed once web-stream guards drop the!isNodeStreamcheck (Node and Web streams have non-overlapping method sets). -
asStream.jsandasWebStream.jssimplified: dropped the inline "is plain value" fast-path (sync-when-possibleprocessValuecovers it), dropped duplicated stop-handler boilerplate, hoisted helpers (unblockDrain,closeReadable,errorReadable,absorbStop). Net:asStream.js271 → 248 lines,asWebStream.js302 → 282 lines. -
Many-array iteration in the slow path no longer allocates an array iterator — walks the values directly with promise-chaining only when needed.
- Tests renamed from
.mjs/.mtsto.js/.ts(the package is now"type": "module", so the explicit extensions are redundant). - New lifecycle test files:
tests/test-asStream-lifecycle.js,tests/test-asWebStream-lifecycle.js,tests/test-streamPuller.js,tests/test-webStreamPuller.js.
- New wiki pages: asWebStream, makeStreamPuller, makeWebStreamPuller, Migration-V3-to-V4.
- Existing pages updated for 4.x: Home (4.x banner, subpath section), chain(), asStream(), defs (generator-yields-plain convention), highWaterMark, utils, fun() (rewritten with memory-caveat + history rationale).
- AI-facing docs (
AGENTS.md,ARCHITECTURE.md,llms.txt,llms-full.txt) fully rewritten for 4.x.
- Special values in generators (don't yield
none/many(...);stop/finalValue(...)supported) — documented in wiki/defs.md § Special values in generators. - If a module declares
export default X, it must also declareexport {X}for the same value. Fleet-wide rule, documented astopics/esm-default-export-with-named-mirrorand slice 17 of thefleet-conventions-bundle.
TypeScript inference fixes for nested and cross-type pipe compositions. Pure type-system patch — no runtime changes. Two distinct bugs found via strict probes of the existing 3.6.2 typings:
Ret<ChainOutput<W, R>> and Arg0<ChainOutput<W, R>> were collapsing to
any because ChainOutput<W, R> = Omit<Duplex, ...> & ChainOutputExtensions<R>
is structurally assignable to Duplex, so the broad
F extends Readable | Transform | Duplex ? any branch in both
Arg0 and Ret matched before any chain-specific check. W was also
not structurally present (the alias parameter was unused in the body).
Fix: mirror the TypedDuplex phantom pattern. Added
__streamTypeW(): W and __streamTypeR(): R phantom methods to
ChainOutputExtensions (now parameterized by both W and R); added
F extends ChainOutput<infer W, any> / F extends ChainOutput<any, infer R>
as the first branch in Arg0 and Ret so chain-of-chain composition
recovers both type parameters.
fun(...fns) returns (arg) => Many<R> | Promise<Many<R>>. When that
function appeared inside another fun, gen, or chain, the outer's
Ret ran OutputType<F> over the union — and UnpackReturnType checked
ReturnType<F> extends Promise<unknown> against the whole union. The
check failed on the Many<R> arm, the unwrap fell through unchanged, and
the inner Promise<Many<R>> leaked downstream as R | Promise<Many<R>>.
For fun-of-fun the pollution compounded to
Many<R | Promise<Many<R>>> | Promise<Many<R | Promise<Many<R>>>>.
Fix: rewrite UnpackReturnType as a thin shim over a naked-T inner alias
that distributes over union members and recurses on Promise<infer U>.
Same form (AsyncGenerator -> Generator -> Promise-recurse -> T) handles
Promise<Promise<X>> and the union arms uniformly.
Every pipe-like API (chain, fun, gen, asStream) is now exercised
in every structurally-permitted position (first / middle / last) of every
other pipe-like API. New tests cover:
- Nested-position guards:
chain([fn, X, fn]),fun(fn, X, fn),gen(fn, X, fn)forX in {fun, gen, asStream, nested chain}. - First-position guards: chain output as first item,
fun(fun-out, fn),gen(gen-out, fn), plus the cross-type pairings. - Threading verification: a deliberately-mismatched-middle repro confirmed
TypeScript pinpoints the mismatch on the middle item via the
ChainList/FnListRet<F1, I>walk.
Typings test count: 56 → 68 (tests/test-typings-chain.mts +
tests/test-typings-gen-fun.mts).
Tarball now includes AGENTS.md and ARCHITECTURE.md alongside the
existing llms.txt / llms-full.txt per the tarball AI-docs
convention — useful for AI tools in downstream
projects discovering the package via npm install.
@types/node ^25.6.0 -> ^25.7.0, nano-benchmark ^1.0.15 -> ^1.0.16,
tape-six ^1.8.0 -> ^1.9.0, tape-six-proc ^1.2.8 -> ^1.2.9.
Per-release cliff-notes reproduced from README.md. Pre-3.6.3 entries
weren't written in the long-form style; the README versions are the
canonical record for those versions.
- 3.6.2 Improved TS typings:
ChainOutput<W, R>propagatesRto events and methods (thx Scover). Updated deps. - 3.6.1 Technical release: updated deps.
- 3.6.0 Performance: sync-first
fun()(~2.5× faster for sync pipelines, now returnsMany | Promise<Many>). Sync fast path ingen()(~1.6× faster). Documentednull/undefinedhandling differences. Wiki: renamed V2 files for Windows compatibility. - 3.5.1 Fixed
finishevent not firing afterstop. Web stream detection uses duck-typing instead ofinstanceof(supports non-standard web streams) (thx Alex Yang). Performance: unified fast path inasStream(). - 3.5.0 Variadic
combineMany()andcombineManyMut(). FixedreadableFrom()unhandled rejection bug. Improved TS typings, docs, andpackage.jsonmetadata. - 3.4.1 Improved TS typings, added TypeScript typing tests, cleaned up dead code.
- 3.4.0 Added
Many-related helpers andchainUnchecked()for TS. - 3.3.2 Technical release: updated deps, more tests.
- 3.3.1 Minor enhancement: more flexible split on lines.
- 3.3.0 Added a way to ignore JSON parsing errors silently.
- 3.2.0 Added TS typings and
clearFunctionList(). - 3.1.0 Added a seamless support for web streams.
- 3.0.1 First release of 3.0. See wiki for details.
- 3.0.0 New major version. Unreleased.
- 2.2.5 Relaxed the definition of a stream (thx Rich Hodgkins).
- 2.2.4 Bugfix: wrong
const-ness in the async generator branch (thx Patrick Pang). - 2.2.3 Technical release. No need to upgrade.
- 2.2.2 Technical release. No need to upgrade.
- 2.2.1 Technical release: new symbols namespace, explicit license (thx Keen Yee Liau), added Greenkeeper.
- 2.2.0 Added utilities:
take,takeWhile,skip,skipWhile,fold,scan,Reduce,comp. - 2.1.0 Added simple transducers, dropped Node 6.
- 2.0.3 Added TypeScript typings and the badge.
- 2.0.2 Workaround for Node 6: use
'finish'event instead of_final(). - 2.0.1 Improved documentation.
- 2.0.0 Upgraded to use Duplex instead of EventEmitter as the base.
- 1.0.3 Improved documentation.
- 1.0.2 Better README.
- 1.0.1 Fixed the README.
- 1.0.0 The initial release.
Start here
API
Transducers
Adapters
I/O & helpers
Tuning & internals
Reference
stream-chain 2.x (legacy)