Skip to content

[XLA] Keep the dot algorithm when importing 2-D dots to StableHLO - #53583

Open
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_1026602448
Open

copybara-service[bot] wants to merge 1 commit into
mainfrom
test_1026602448

Conversation

@copybara-service

Copy link
Copy Markdown

[XLA] Keep the dot algorithm when importing 2-D dots to StableHLO

Why

A sharded jax.jit program partitioned with Shardy (the default partitioner)
that calls jnp.dot(a, b, precision=jax.lax.DotAlgorithmPreset.F32_F32_F32)
on 2-D operands silently runs at default precision. On GPU that is TF32: the
relative error is 2.9e-4 instead of 2.9e-7. TF32_TF32_F32_X3 and
BF16_BF16_F32_X6 show the same error. With GSPMD the results are correct.
Reported in #53163.

Root cause

Shardy round-trips the module through StableHLO. On import,
HloFunctionImporter turns a dot with default dimension numbers (vector or
matrix operands, no batch dimensions) into stablehlo.dot. stablehlo.dot has
no algorithm attribute, so the algorithm is only kept as a discardable
attribute, and the export to HLO drops it. Dots with batch dimensions already
import as stablehlo.dot_general, which keeps the algorithm.

Fix

DotIsDefault now returns false for a dot with an algorithm. Such dots take the
existing stablehlo.dot_general path, the same path batched algorithm dots
already use. Dots without an algorithm still import as stablehlo.dot. A 2-D
dot with an algorithm still exports to an HLO dot with the same dimension
numbers, and now keeps its algorithm.

Testing

Added xla/hlo/translate/tests/dot_algorithm_import.hlo. It checks that a 2-D
dot with algorithm=dot_f32_f32_f32 imports as stablehlo.dot_general with the
algorithm, and that the algorithm is still there after HLO -> StableHLO -> HLO.
The test fails without the fix.

**Why**

A sharded `jax.jit` program partitioned with Shardy (the default partitioner)
that calls `jnp.dot(a, b, precision=jax.lax.DotAlgorithmPreset.F32_F32_F32)`
on 2-D operands silently runs at default precision. On GPU that is TF32: the
relative error is 2.9e-4 instead of 2.9e-7. `TF32_TF32_F32_X3` and
`BF16_BF16_F32_X6` show the same error. With GSPMD the results are correct.
Reported in #53163.

**Root cause**

Shardy round-trips the module through StableHLO. On import,
`HloFunctionImporter` turns a dot with default dimension numbers (vector or
matrix operands, no batch dimensions) into `stablehlo.dot`. `stablehlo.dot` has
no `algorithm` attribute, so the algorithm is only kept as a discardable
attribute, and the export to HLO drops it. Dots with batch dimensions already
import as `stablehlo.dot_general`, which keeps the algorithm.

**Fix**

`DotIsDefault` now returns false for a dot with an algorithm. Such dots take the
existing `stablehlo.dot_general` path, the same path batched algorithm dots
already use. Dots without an algorithm still import as `stablehlo.dot`. A 2-D
dot with an algorithm still exports to an HLO `dot` with the same dimension
numbers, and now keeps its algorithm.

**Testing**

Added `xla/hlo/translate/tests/dot_algorithm_import.hlo`. It checks that a 2-D
dot with `algorithm=dot_f32_f32_f32` imports as `stablehlo.dot_general` with the
algorithm, and that the algorithm is still there after HLO -> StableHLO -> HLO.
The test fails without the fix.

PiperOrigin-RevId: 1026602448
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

⌂ 换网址