Repository navigation
[XLA] Keep the dot algorithm when importing 2-D dots to StableHLO - #53583
Open
copybara-service[bot] wants to merge 1 commit into
Open
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
**Why** A sharded `jax.jit` program partitioned with Shardy (the default partitioner) that calls `jnp.dot(a, b, precision=jax.lax.DotAlgorithmPreset.F32_F32_F32)` on 2-D operands silently runs at default precision. On GPU that is TF32: the relative error is 2.9e-4 instead of 2.9e-7. `TF32_TF32_F32_X3` and `BF16_BF16_F32_X6` show the same error. With GSPMD the results are correct. Reported in #53163. **Root cause** Shardy round-trips the module through StableHLO. On import, `HloFunctionImporter` turns a dot with default dimension numbers (vector or matrix operands, no batch dimensions) into `stablehlo.dot`. `stablehlo.dot` has no `algorithm` attribute, so the algorithm is only kept as a discardable attribute, and the export to HLO drops it. Dots with batch dimensions already import as `stablehlo.dot_general`, which keeps the algorithm. **Fix** `DotIsDefault` now returns false for a dot with an algorithm. Such dots take the existing `stablehlo.dot_general` path, the same path batched algorithm dots already use. Dots without an algorithm still import as `stablehlo.dot`. A 2-D dot with an algorithm still exports to an HLO `dot` with the same dimension numbers, and now keeps its algorithm. **Testing** Added `xla/hlo/translate/tests/dot_algorithm_import.hlo`. It checks that a 2-D dot with `algorithm=dot_f32_f32_f32` imports as `stablehlo.dot_general` with the algorithm, and that the algorithm is still there after HLO -> StableHLO -> HLO. The test fails without the fix. PiperOrigin-RevId: 1026602448
copybara-service
Bot
force-pushed
the
test_1026602448
branch
from
November 20, 2026 19:13
f821ac5 to
eb482a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[XLA] Keep the dot algorithm when importing 2-D dots to StableHLO
Why
A sharded
jax.jitprogram partitioned with Shardy (the default partitioner)that calls
jnp.dot(a, b, precision=jax.lax.DotAlgorithmPreset.F32_F32_F32)on 2-D operands silently runs at default precision. On GPU that is TF32: the
relative error is 2.9e-4 instead of 2.9e-7.
TF32_TF32_F32_X3andBF16_BF16_F32_X6show the same error. With GSPMD the results are correct.Reported in #53163.
Root cause
Shardy round-trips the module through StableHLO. On import,
HloFunctionImporterturns a dot with default dimension numbers (vector ormatrix operands, no batch dimensions) into
stablehlo.dot.stablehlo.dothasno
algorithmattribute, so the algorithm is only kept as a discardableattribute, and the export to HLO drops it. Dots with batch dimensions already
import as
stablehlo.dot_general, which keeps the algorithm.Fix
DotIsDefaultnow returns false for a dot with an algorithm. Such dots take theexisting
stablehlo.dot_generalpath, the same path batched algorithm dotsalready use. Dots without an algorithm still import as
stablehlo.dot. A 2-Ddot with an algorithm still exports to an HLO
dotwith the same dimensionnumbers, and now keeps its algorithm.
Testing
Added
xla/hlo/translate/tests/dot_algorithm_import.hlo. It checks that a 2-Ddot with
algorithm=dot_f32_f32_f32imports asstablehlo.dot_generalwith thealgorithm, and that the algorithm is still there after HLO -> StableHLO -> HLO.
The test fails without the fix.