Skip to content

[Redis 8.10] Add FT.AGGREGATE REDUCE COLLECT support with complex-value FieldValue - #3878

Merged
a-TODO-rov merged 3 commits into
mainfrom
feat/ft-aggregate-collect-reducer-v2
Sep 24, 2026
Merged

a-TODO-rov merged 3 commits into
mainfrom
feat/ft-aggregate-collect-reducer-v2

Conversation

@uglide

@uglide uglide commented Jul 31, 2026 •

Copy link
Copy Markdown
Collaborator

Adds the experimental REDUCE COLLECT reducer for FT.AGGREGATE, which gathers per-row field projections within each GROUPBY group. To carry its nested column values, FieldValue is extended from a scalar-only wrapper into a tagged union (Kind: SCALAR, ARRAY, MAP, NULL), and SearchReplyParser decodes nested reply structures recursively instead of failing on the ByteBuffer cast.

Key Decisions & Assumptions

  • Stacked on fix: rework search API generics to replace unnecessary type params with String #3716; the FieldValue extension assumes that reply model and should be rebased if it changes.
  • Parsing keeps raw protocol shapes (a RESP2 collected entry stays ARRAY, RESP3 MAP); normalization happens at read time via a lenient asMap() that interprets a flat even-length array as key/value pairs. Callers read collected columns uniformly as asList() → per-entry asMap() on both protocols. Any even-length scalar array (for example a TOLIST column) is accepted by this view; documented as caller responsibility.
  • COLLECT stays in AggregateArgs (the aggregateutils package serves FT.HYBRID only).

Behavioral / Conceptual Changes

  • Nested reply values (COLLECT, TOLIST) now parse into FieldValue arrays/maps; previously the parser failed on them. Scalar columns decode byte-identically to before.
  • Accessing a FieldValue through the wrong kind (for example asString() on an array) throws IllegalStateException; an odd-length pair array is rejected rather than silently truncated.
  • The builder validates COLLECT usage: FIELDS * and explicit fields are mutually exclusive, and a reducer without fields fails at encode time.

Testing

New FieldValueUnitTests cover kind discrimination, the pair-normalizing asMap(), immutability, and structural equality. SearchReplyCollectParserUnitTests verify RESP2/RESP3 collected-column parsing produces equal normalized entries, null preservation inside entries, and TOLIST-style arrays. Command-builder tests assert the exact REDUCE COLLECT wire format; an integration test runs a full COLLECT aggregation, self-skipping on servers without search-enable-unstable-features.

Notes

  • COLLECT is gated behind search-enable-unstable-features; both the server feature and this API are marked @Experimental.
  • HybridReplyParser still decodes field values as scalars only; FT.HYBRID cannot emit COLLECT today, so this is a known limitation, not a regression.

Note

Medium Risk
Extends a core search reply type and parser; scalar behavior is preserved but new kind checks can surface IllegalStateException if callers use scalar accessors on nested aggregation columns.

Overview
Adds experimental FT.AGGREGATE REDUCE COLLECT support via AggregateArgs.Reducer.collect() and a CollectReducer builder (fields / fieldsAll, in-group sortBy, limit, as), marked @Experimental and documented as requiring search-enable-unstable-features.

FieldValue is no longer scalar-only: it introduces Kind (SCALAR, ARRAY, MAP, NULL), factories (array, map, nullValue), and asList() / asMap() with kind checks (IllegalStateException on mismatch). asMap() normalizes RESP2 flat key/value arrays and RESP3 maps so collected entries read the same on both protocols.

SearchReplyParser recursively decodes nested reply shapes (COLLECT, TOLIST) into nested FieldValues instead of assuming every value is a ByteBuffer. Scalar columns still decode as before; wrong accessors on complex columns now fail explicitly.

User guide adds a COLLECT section with usage examples. Tests cover wire encoding, parser behavior on RESP2/RESP3, FieldValue semantics, and an integration path that skips when the server cannot enable unstable features.

Reviewed by Cursor Bugbot for commit a1ec900. Bugbot is set up for automated code reviews on this repo. Configure here.

@uglide uglide changed the title [Redis 8.10]Add support ft aggregate collect reducer [Redis 8.10] Add FT.AGGREGATE REDUCE COLLECT support with complex-value FieldValue Jul 31, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 03a3031e49

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@uglide
uglide changed the base branch from main to rework-codecs-for-redis-search-module August 19, 2026 11:50
@a-TODO-rov
a-TODO-rov force-pushed the rework-codecs-for-redis-search-module branch from b6d2ec3 to a605ccf Compare August 24, 2026 12:59
@viktoriya-kutsarova
viktoriya-kutsarova force-pushed the rework-codecs-for-redis-search-module branch 4 times, most recently from b33a00b to 2149635 Compare September 4, 2026 21:58
Base automatically changed from rework-codecs-for-redis-search-module to main September 10, 2026 10:56
@uglide
uglide force-pushed the feat/ft-aggregate-collect-reducer-v2 branch from 03a3031 to 96806b8 Compare September 23, 2026 13:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-24T12:28:31.168171Z a1ec900 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@uglide
uglide requested a review from a-TODO-rov September 23, 2026 13:29

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 96806b869a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- Make SearchResult#addField(String, FieldValue) package-private so the public
  addField(String, byte[]) stays unambiguous for null arguments; the reply
  parsers in the same package are the only producers of nested FieldValue
  columns.
- Add @SInCE 7.8 to the new COLLECT API (Reducer#collect() and the
  CollectReducer fluent methods) and document the contract exceptions they
  throw.
- Add @SInCE 7.8 to the FieldValue members that were still untagged.
- Gate shouldPerformCollectAggregation on Redis 8.10 instead of catching every
  RedisCommandExecutionException from FT.AGGREGATE, so a builder regression
  fails the test rather than silently skipping it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6dbe3c3c67

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a1ec9000f0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1274 to +1275
@Override
public CollectReducer as(String alias) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add a release tag to the covariant as override

The earlier @since fix covered the other fluent methods, but CollectReducer.as(String) is also a newly declared public API with a covariant return type and still has no @since 7.8 tag; Javadoc does not inherit this tag from Reducer.as(String), so the generated API omits its introduction version. Add an override doc comment with the release tag.

AGENTS.md reference: AGENTS.md:L149-L150

Useful? React with 👍 / 👎.

@a-TODO-rov a-TODO-rov left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@a-TODO-rov
a-TODO-rov merged commit e5a7857 into main Sep 24, 2026
10 checks passed
@a-TODO-rov
a-TODO-rov deleted the feat/ft-aggregate-collect-reducer-v2 branch September 24, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

⌂ 换网址