Skip to content

PR #52240: [XLA:GPU] Preserve frontend attributes in dot rewriters - #52759

Closed
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_1018979554
Closed

copybara-service[bot] wants to merge 1 commit into
mainfrom
test_1018979554

Conversation

@copybara-service

Copy link
Copy Markdown

PR #52240: [XLA:GPU] Preserve frontend attributes in dot rewriters

Imported from GitHub PR #52240

📝 Summary of Changes

Preserve frontend attributes when dot rewriters replace a dot with new instructions. New dots are added with HloInstruction::AddInstruction on the original dot, which derives them via SetupDerivedInstruction (also carrying metadata and, where shapes allow, sharding, like existing rewriters such as DotDimensionSorter), so they inherit the original dot's frontend attributes instead of silently dropping them. This covers DotAlgorithmRewriter (the BF16/TF32 decompositions in backends/gpu/transforms and the FP8xN decomposition in hlo/transforms/dot_algorithm_rewriter.cc), ScaledDotRewriter, SplitkRewriter, and GemmFusionSwapOperands.

🎯 Justification

Frontend attributes are the mechanism by which the frontend (e.g. JAX) attaches annotations to HLO instructions for downstream passes to consume. When a pass replaces an instruction, it must carry those attributes over to the replacement; otherwise the annotation is silently dropped and the consumers that depend on it no longer see it. These dot rewriters were not doing this, so annotated matmuls lost their scheduling annotations.

🧭 Design: deriving replacements from the original dot

Each new dot is added with original_dot->AddInstruction(...), which derives it via SetupDerivedInstruction (metadata, frontend attributes, and sharding where the shapes allow). Instructions built by helpers call SetupDerivedInstruction directly: the instruction that replaces the dot in RewriteDotWithOperandPairs (shared by BF16X3/X6/X9 and TF32X3) and RewriteFp8xNDot (FP8X3/X4), and the split-K dot, which MakeDotHlo creates. So every new dot and the instruction that replaces the original keep the annotation.

🚀 Kind of Contribution

🐛 Bug Fix

🧪 Unit Tests:

Added hardware-independent tests asserting frontend attributes survive each dot rewrite:

  • backends/gpu/transforms/dot_algorithm_rewriter_test.cc:
    • KeepsFrontendAttributes, parameterized over every PrecisionConfig::Algorithm: verifies that after the pass every dot and the instruction that replaces the dot keep the attributes. Without the fix it fails for the seven algorithms the pass decomposes (BF16, BF16X3/X6/X9, TF32X3, FP8X3/X4); new algorithms are covered automatically.
  • hlo/transforms/dot_algorithm_rewriter_test.cc:
    • RewriteFp8xNDotStructureTest.KeepsFrontendAttributes, for FP8X3 and FP8X4 at both precisions: verifies every F8 dot and the instruction that replaces the dot keep them.
  • gemm_fusion_swap_operands_test.cc:
    • SwappedDotKeepsFrontendAttributes: verifies the swapped dot keeps them.
  • scaled_dot_rewriter_test.cc:
    • RewrittenDotKeepsFrontendAttributes: verifies the dot replacing a scaled-dot keeps them.
  • splitk_rewriter_test.cc:
    • SplitKDotKeepsFrontendAttributes: verifies the split-K dot keeps them.

🧪 Execution Tests:

No new checked-in execution test was added.
Copybara import of the project:

--
a3f38cf by Jessica De Silva jdesilva@nvidia.com:

[XLA:GPU] Preserve frontend attributes in dot rewriters

Dot rewriters that replace a dot with new instructions now derive the new dots
and the replacement via SetupDerivedInstruction, so they inherit the original
dot's metadata and frontend attributes instead of silently dropping them. This
covers DotAlgorithmRewriter (including the FP8xN decomposition in
hlo/transforms/dot_algorithm_rewriter.cc), ScaledDotRewriter, SplitkRewriter,
and GemmFusionSwapOperands.

Merging this change closes #52240

FUTURE_COPYBARA_INTEGRATE_REVIEW=#52240 from NVIDIA:nv/jessica-desilva/dot-rewriters-keep-frontend-attributes-rewrite a3f38cf

@copybara-service
copybara-service Bot force-pushed the test_1018979554 branch 2 times, most recently from b877b2e to c605579 Compare November 18, 2026 09:30
Imported from GitHub PR #52240

📝 **Summary of Changes**

Preserve frontend attributes when dot rewriters replace a dot with new instructions. New dots are added with `HloInstruction::AddInstruction` on the original dot, which derives them via `SetupDerivedInstruction` (also carrying metadata and, where shapes allow, sharding, like existing rewriters such as `DotDimensionSorter`), so they inherit the original dot's frontend attributes instead of silently dropping them. This covers `DotAlgorithmRewriter` (the BF16/TF32 decompositions in `backends/gpu/transforms` and the FP8xN decomposition in `hlo/transforms/dot_algorithm_rewriter.cc`), `ScaledDotRewriter`, `SplitkRewriter`, and `GemmFusionSwapOperands`.

🎯 **Justification**

Frontend attributes are the mechanism by which the frontend (e.g. JAX) attaches annotations to HLO instructions for downstream passes to consume. When a pass replaces an instruction, it must carry those attributes over to the replacement; otherwise the annotation is silently dropped and the consumers that depend on it no longer see it. These dot rewriters were not doing this, so annotated matmuls lost their scheduling annotations.

🧭 **Design: deriving replacements from the original dot**

Each new dot is added with `original_dot->AddInstruction(...)`, which derives it via `SetupDerivedInstruction` (metadata, frontend attributes, and sharding where the shapes allow). Instructions built by helpers call `SetupDerivedInstruction` directly: the instruction that replaces the dot in `RewriteDotWithOperandPairs` (shared by BF16X3/X6/X9 and TF32X3) and `RewriteFp8xNDot` (FP8X3/X4), and the split-K dot, which `MakeDotHlo` creates. So every new dot and the instruction that replaces the original keep the annotation.

🚀 **Kind of Contribution**

🐛 Bug Fix

🧪 **Unit Tests:**

Added hardware-independent tests asserting frontend attributes survive each dot rewrite:

* `backends/gpu/transforms/dot_algorithm_rewriter_test.cc`:
  * `KeepsFrontendAttributes`, parameterized over every `PrecisionConfig::Algorithm`: verifies that after the pass every dot and the instruction that replaces the dot keep the attributes. Without the fix it fails for the seven algorithms the pass decomposes (BF16, BF16X3/X6/X9, TF32X3, FP8X3/X4); new algorithms are covered automatically.
* `hlo/transforms/dot_algorithm_rewriter_test.cc`:
  * `RewriteFp8xNDotStructureTest.KeepsFrontendAttributes`, for FP8X3 and FP8X4 at both precisions: verifies every F8 dot and the instruction that replaces the dot keep them.
* `gemm_fusion_swap_operands_test.cc`:
  * `SwappedDotKeepsFrontendAttributes`: verifies the swapped dot keeps them.
* `scaled_dot_rewriter_test.cc`:
  * `RewrittenDotKeepsFrontendAttributes`: verifies the dot replacing a scaled-dot keeps them.
* `splitk_rewriter_test.cc`:
  * `SplitKDotKeepsFrontendAttributes`: verifies the split-K dot keeps them.

🧪 **Execution Tests:**

No new checked-in execution test was added.
Copybara import of the project:

--
a3f38cf by Jessica De Silva <jdesilva@nvidia.com>:

[XLA:GPU] Preserve frontend attributes in dot rewriters

Dot rewriters that replace a dot with new instructions now derive the new dots
and the replacement via SetupDerivedInstruction, so they inherit the original
dot's metadata and frontend attributes instead of silently dropping them. This
covers DotAlgorithmRewriter (including the FP8xN decomposition in
hlo/transforms/dot_algorithm_rewriter.cc), ScaledDotRewriter, SplitkRewriter,
and GemmFusionSwapOperands.

Merging this change closes #52240

FUTURE_COPYBARA_INTEGRATE_REVIEW=#52240 from NVIDIA:nv/jessica-desilva/dot-rewriters-keep-frontend-attributes-rewrite a3f38cf
PiperOrigin-RevId: 1018979554
@copybara-service copybara-service Bot closed this Nov 20, 2026
@copybara-service
copybara-service Bot deleted the test_1018979554 branch November 20, 2026 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

⌂ 换网址