Repository navigation
PR #52240: [XLA:GPU] Preserve frontend attributes in dot rewriters - #52759
Closed
copybara-service[bot] wants to merge 1 commit into
Closed
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
copybara-service
Bot
force-pushed
the
test_1018979554
branch
2 times, most recently
from
November 18, 2026 09:30
b877b2e to
c605579
Compare
Imported from GitHub PR #52240 📝 **Summary of Changes** Preserve frontend attributes when dot rewriters replace a dot with new instructions. New dots are added with `HloInstruction::AddInstruction` on the original dot, which derives them via `SetupDerivedInstruction` (also carrying metadata and, where shapes allow, sharding, like existing rewriters such as `DotDimensionSorter`), so they inherit the original dot's frontend attributes instead of silently dropping them. This covers `DotAlgorithmRewriter` (the BF16/TF32 decompositions in `backends/gpu/transforms` and the FP8xN decomposition in `hlo/transforms/dot_algorithm_rewriter.cc`), `ScaledDotRewriter`, `SplitkRewriter`, and `GemmFusionSwapOperands`. 🎯 **Justification** Frontend attributes are the mechanism by which the frontend (e.g. JAX) attaches annotations to HLO instructions for downstream passes to consume. When a pass replaces an instruction, it must carry those attributes over to the replacement; otherwise the annotation is silently dropped and the consumers that depend on it no longer see it. These dot rewriters were not doing this, so annotated matmuls lost their scheduling annotations. 🧭 **Design: deriving replacements from the original dot** Each new dot is added with `original_dot->AddInstruction(...)`, which derives it via `SetupDerivedInstruction` (metadata, frontend attributes, and sharding where the shapes allow). Instructions built by helpers call `SetupDerivedInstruction` directly: the instruction that replaces the dot in `RewriteDotWithOperandPairs` (shared by BF16X3/X6/X9 and TF32X3) and `RewriteFp8xNDot` (FP8X3/X4), and the split-K dot, which `MakeDotHlo` creates. So every new dot and the instruction that replaces the original keep the annotation. 🚀 **Kind of Contribution** 🐛 Bug Fix 🧪 **Unit Tests:** Added hardware-independent tests asserting frontend attributes survive each dot rewrite: * `backends/gpu/transforms/dot_algorithm_rewriter_test.cc`: * `KeepsFrontendAttributes`, parameterized over every `PrecisionConfig::Algorithm`: verifies that after the pass every dot and the instruction that replaces the dot keep the attributes. Without the fix it fails for the seven algorithms the pass decomposes (BF16, BF16X3/X6/X9, TF32X3, FP8X3/X4); new algorithms are covered automatically. * `hlo/transforms/dot_algorithm_rewriter_test.cc`: * `RewriteFp8xNDotStructureTest.KeepsFrontendAttributes`, for FP8X3 and FP8X4 at both precisions: verifies every F8 dot and the instruction that replaces the dot keep them. * `gemm_fusion_swap_operands_test.cc`: * `SwappedDotKeepsFrontendAttributes`: verifies the swapped dot keeps them. * `scaled_dot_rewriter_test.cc`: * `RewrittenDotKeepsFrontendAttributes`: verifies the dot replacing a scaled-dot keeps them. * `splitk_rewriter_test.cc`: * `SplitKDotKeepsFrontendAttributes`: verifies the split-K dot keeps them. 🧪 **Execution Tests:** No new checked-in execution test was added. Copybara import of the project: -- a3f38cf by Jessica De Silva <jdesilva@nvidia.com>: [XLA:GPU] Preserve frontend attributes in dot rewriters Dot rewriters that replace a dot with new instructions now derive the new dots and the replacement via SetupDerivedInstruction, so they inherit the original dot's metadata and frontend attributes instead of silently dropping them. This covers DotAlgorithmRewriter (including the FP8xN decomposition in hlo/transforms/dot_algorithm_rewriter.cc), ScaledDotRewriter, SplitkRewriter, and GemmFusionSwapOperands. Merging this change closes #52240 FUTURE_COPYBARA_INTEGRATE_REVIEW=#52240 from NVIDIA:nv/jessica-desilva/dot-rewriters-keep-frontend-attributes-rewrite a3f38cf PiperOrigin-RevId: 1018979554
copybara-service
Bot
force-pushed
the
test_1018979554
branch
from
November 18, 2026 09:58
c605579 to
42bc647
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR #52240: [XLA:GPU] Preserve frontend attributes in dot rewriters
Imported from GitHub PR #52240
📝 Summary of Changes
Preserve frontend attributes when dot rewriters replace a dot with new instructions. New dots are added with
HloInstruction::AddInstructionon the original dot, which derives them viaSetupDerivedInstruction(also carrying metadata and, where shapes allow, sharding, like existing rewriters such asDotDimensionSorter), so they inherit the original dot's frontend attributes instead of silently dropping them. This coversDotAlgorithmRewriter(the BF16/TF32 decompositions inbackends/gpu/transformsand the FP8xN decomposition inhlo/transforms/dot_algorithm_rewriter.cc),ScaledDotRewriter,SplitkRewriter, andGemmFusionSwapOperands.🎯 Justification
Frontend attributes are the mechanism by which the frontend (e.g. JAX) attaches annotations to HLO instructions for downstream passes to consume. When a pass replaces an instruction, it must carry those attributes over to the replacement; otherwise the annotation is silently dropped and the consumers that depend on it no longer see it. These dot rewriters were not doing this, so annotated matmuls lost their scheduling annotations.
🧭 Design: deriving replacements from the original dot
Each new dot is added with
original_dot->AddInstruction(...), which derives it viaSetupDerivedInstruction(metadata, frontend attributes, and sharding where the shapes allow). Instructions built by helpers callSetupDerivedInstructiondirectly: the instruction that replaces the dot inRewriteDotWithOperandPairs(shared by BF16X3/X6/X9 and TF32X3) andRewriteFp8xNDot(FP8X3/X4), and the split-K dot, whichMakeDotHlocreates. So every new dot and the instruction that replaces the original keep the annotation.🚀 Kind of Contribution
🐛 Bug Fix
🧪 Unit Tests:
Added hardware-independent tests asserting frontend attributes survive each dot rewrite:
backends/gpu/transforms/dot_algorithm_rewriter_test.cc:KeepsFrontendAttributes, parameterized over everyPrecisionConfig::Algorithm: verifies that after the pass every dot and the instruction that replaces the dot keep the attributes. Without the fix it fails for the seven algorithms the pass decomposes (BF16, BF16X3/X6/X9, TF32X3, FP8X3/X4); new algorithms are covered automatically.hlo/transforms/dot_algorithm_rewriter_test.cc:RewriteFp8xNDotStructureTest.KeepsFrontendAttributes, for FP8X3 and FP8X4 at both precisions: verifies every F8 dot and the instruction that replaces the dot keep them.gemm_fusion_swap_operands_test.cc:SwappedDotKeepsFrontendAttributes: verifies the swapped dot keeps them.scaled_dot_rewriter_test.cc:RewrittenDotKeepsFrontendAttributes: verifies the dot replacing a scaled-dot keeps them.splitk_rewriter_test.cc:SplitKDotKeepsFrontendAttributes: verifies the split-K dot keeps them.🧪 Execution Tests:
No new checked-in execution test was added.
Copybara import of the project:
--
a3f38cf by Jessica De Silva jdesilva@nvidia.com:
[XLA:GPU] Preserve frontend attributes in dot rewriters
Dot rewriters that replace a dot with new instructions now derive the new dots
and the replacement via SetupDerivedInstruction, so they inherit the original
dot's metadata and frontend attributes instead of silently dropping them. This
covers DotAlgorithmRewriter (including the FP8xN decomposition in
hlo/transforms/dot_algorithm_rewriter.cc), ScaledDotRewriter, SplitkRewriter,
and GemmFusionSwapOperands.
Merging this change closes #52240
FUTURE_COPYBARA_INTEGRATE_REVIEW=#52240 from NVIDIA:nv/jessica-desilva/dot-rewriters-keep-frontend-attributes-rewrite a3f38cf