Skip to content

Cranelift/Wasmtime/Pulley/Debugging: use little-endian mode to spill/reload vectors in guest-debugging slot and ABI clobbers.#12585

Merged
cfallin merged 3 commits intobytecodealliance:mainfrom
cfallin:pulley-little-endian-vecs-on-big-endian-when-debugging
Feb 12, 2026
Merged

Cranelift/Wasmtime/Pulley/Debugging: use little-endian mode to spill/reload vectors in guest-debugging slot and ABI clobbers.#12585
cfallin merged 3 commits intobytecodealliance:mainfrom
cfallin:pulley-little-endian-vecs-on-big-endian-when-debugging

Conversation

@cfallin
Copy link
Member

@cfallin cfallin commented Feb 12, 2026

When running Pulley on an s390x (or other big-endian) host, and enabling guest-debugging instrumentation, a very strange confluence of events occurs:

  • Pulley uses "native endian" of the host by default for loads and stores.
  • Patchable calls to debug hooks use the preserve_all ABI, which spills all registers in the trampoline adapter (callee in this ABI), including vector registers.
  • Saving vector-typed locals/operand stack values to the debugger state slot also uses vector stores.
  • All of these stores were thus big-endian on big-endian hosts.
  • Pulley's bytecode only supports little-endian vector loads/stores.

We were thus hitting an assert in Pulley codegen (the Cranelift backend) when encountering a VStore VCode instruction with a big-endian mode.

This PR makes two changes that avoid this issue:

  • The ABI code for Pulley is careful to specify little-endian mode explicitly for any vector load/store.
  • The debug instrumentation code is refactored to use little-endian explicitly for vector types only.
    • (Why not for all types? Because we GC-root GC ref values, and these need to be provided to the collector as mutable storage cells, so need to be in native endianness.)

Test will come as part of #12575 incorporating a
Pulley-with-guest-debugging test and running on s390x amongst our platforms.

@cfallin cfallin requested review from a team as code owners February 12, 2026 21:47
@cfallin cfallin requested review from fitzgen and removed request for a team February 12, 2026 21:47
…reload vectors in guest-debugging slot and ABI clobbers.

When running Pulley on an s390x (or other big-endian) host, and enabling
guest-debugging instrumentation, a very strange confluence of events
occurs:

- Pulley uses "native endian" of the host by default for loads and
  stores.
- Patchable calls to debug hooks use the `preserve_all` ABI, which
  spills all registers in the trampoline adapter (callee in this ABI),
  including vector registers.
- Saving vector-typed locals/operand stack values to the debugger state
  slot also uses vector stores.
- All of these stores were thus big-endian on big-endian hosts.
- Pulley's bytecode only supports little-endian vector loads/stores.

We were thus hitting an assert in Pulley codegen (the Cranelift
backend) when encountering a `VStore` VCode instruction with a
big-endian mode.

This PR makes two changes that avoid this issue:

- The ABI code for Pulley is careful to specify little-endian mode
  explicitly for any vector load/store.
- The debug instrumentation code is refactored to use little-endian
  explicitly for vector types *only*.
  - (Why not for all types? Because we GC-root GC ref values, and
    these need to be provided to the collector as mutable storage
    cells, so need to be in native endianness.)

Test will come as part of bytecodealliance#12575 incorporating a
Pulley-with-guest-debugging test and running on s390x amongst our
platforms.

prtest:full
@cfallin cfallin force-pushed the pulley-little-endian-vecs-on-big-endian-when-debugging branch from d8c01da to a40556a Compare February 12, 2026 21:48
@cfallin cfallin enabled auto-merge February 12, 2026 22:27
@cfallin cfallin added this pull request to the merge queue Feb 12, 2026
Merged via the queue into bytecodealliance:main with commit d9038ed Feb 12, 2026
174 checks passed
@cfallin cfallin deleted the pulley-little-endian-vecs-on-big-endian-when-debugging branch February 12, 2026 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments