Skip to content

ScanIterator.scan.stream().toList() causes "java.lang.IllegalStateException: Accept exceeded fixed size of 0" on Java 16+ #3002

Closed

Description

@lwestby-chwy

Bug Report

Current Behavior

Attempting to call toList() (a new stream operation as of Java 16) on a stream obtained from a ScanIterator throws an IllegalStateException originating from several layers deep within the JDK. Tested on both Java 17 and Java 21.

This appears to match the description for a similar issue encountered in Spring, if this helps: spring-projects/spring-data-commons#2519. In their case, it has to do with the size or characteristic passed along on creation of the stream leading to a FixedNodeBuilder expecting 0 elements.

The older .collect(Collectors.toList()) operation does not encounter this issue.

Stack trace
java.lang.IllegalStateException: Accept exceeded fixed size of 0
	at java.base/java.util.stream.Nodes$FixedNodeBuilder.accept(Nodes.java:1224)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
	at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
	at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
	at io.lettuce.core.ScanIteratorIntegrationTests.testScanIteratorStream(ScanIteratorIntegrationTests.java:85)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Input Code

Input Code

Reproducible right within ScanIteratorIntegrationTests.java if running on Java 16 JRE or above:

@Test
void testScanIteratorStream() {
    redis.mset(KeysAndValues.MAP);

    List<String> result = ScanIterator.scan(redis, ScanArgs.Builder.limit(50).match("*"))
        .stream()
        .toList();
}

Expected behavior/code

The toList() operation returns a list successfully.

Environment

  • Lettuce version(s): 6.3.2.RELEASE as well as latest main branch
  • Redis version: latest Redis 7 on DockerHub

Possible Solution

spring-projects/spring-data-commons#2519 points to the commit where they fixed it, although I can't pretend I understand how much of the full commit is required to solve this here.

Updating ScanIterator.stream() to set the size to -1 on the spliterator does fix it locally, but not sure if there are knock-on effects that require more code:

public Stream<T> stream() {
    return StreamSupport.stream(Spliterators.spliterator(this, -1, 0), false);
}

Additional context

Java version info:

Java 21:

openjdk version "21.0.3" 2024-04-16 LTS
OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode)

Java 17:

openjdk version "17.0.11" 2024-04-16
OpenJDK Runtime Environment Temurin-17.0.11+9 (build 17.0.11+9)
OpenJDK 64-Bit Server VM Temurin-17.0.11+9 (build 17.0.11+9, mixed mode)

Activity

  1. added this to the 6.5.0.RELEASE milestone on Oct 18, 2024
  2. tishun commented on Oct 18, 2024

    @tishun
    Collaborator

    Hey @lwestby-chwy,
    thanks for the detail analysis and report. IMHO the fix from the spring-data-commons repo should be applied here too. I will leave @mp911de comment if he thinks this is wrong, but as far as I can understand the problem is similar.

    I will prepare a PR and try to get it in the 6.5.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      ⌂ 换网址