[ET Device Support] Device-aware memory planning: separate buffers per device type by Gasoonjia · Pull Request #18375 · pytorch/executorch

Gasoonjia · 2026-03-20T18:21:59Z

Stack from ghstack (oldest at bottom):

-> [ET Device Support] Device-aware memory planning: separate buffers per device type #18375
[ET Device Support] Add NonConstBufferDevice schema for per-buffer device mapping #18330
[ET Device Support] DeviceAllocator interface and DeviceAllocatorRegistry #17535
[ET Device Support] Annotate device attributes of CUDA backend IO tensors cuda device #18080
[ET Device Support] Parse device info from serialized tensor in tensor_parser #18328
[ET Device Support] Propagate device info from TensorSpec into serialized Tensor #18079
[executorch] Propagate device metadata from partitioner result onto TensorSpecs #18078
[ET Device Support] TensorImpl carries device info #17534
[ET Device Support] Schema changes: device info on Tensor and buffer-level device array #17533

Extends memory planning to separate device tensors from CPU tensors into distinct
memory buffers. Non-CPU TensorSpecs (e.g., CUDA) are pre-assigned device-specific
mem_ids before the greedy/naive algorithm runs, ensuring they get planned into
independent memory buffers that never share space with CPU tensors.

Differential Revision: D97447105

…r device type Extends memory planning to separate device tensors from CPU tensors into distinct memory buffers. Non-CPU TensorSpecs (e.g., CUDA) are pre-assigned device-specific mem_ids before the greedy/naive algorithm runs, ensuring they get planned into independent memory buffers that never share space with CPU tensors. Differential Revision: [D97447105](https://our.internmc.facebook.com/intern/diff/D97447105/) [ghstack-poisoned]

pytorch-bot · 2026-03-20T18:22:04Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18375

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 62 New Failures, 1 Cancelled Job, 4 Pending

As of commit 747dbaa with merge base 38b40bc ():

NEW FAILURES - The following jobs have failed:

Build documentation / build (buck2) / Build doc (gh)
RuntimeError: Command docker exec -t ae0adf47e78cfaad5a88a95350a4f5ce188f9603a0677adb0e5b94e275c67279 /exec failed with exit code 2
periodic / test-models-linux (buck2, mv3, portable, linux.2xlarge, 90) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
periodic / test-models-linux (buck2, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
periodic / test-models-linux (cmake, mv3, portable, linux.2xlarge, 90) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
periodic / test-models-linux (cmake, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
periodic / test-models-linux (cmake, vit, portable, linux.2xlarge, 90) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
periodic / test-models-linux (cmake, vit, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-build-wasm-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-lora-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-lora-multimethod-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-mcu-cortex-m-backend / linux-job (gh)
RuntimeError: Command docker exec -t 91df0ba4e1312b50b888f10ca5549e2586c5a8bed3dc29378eb94dce948d1cbb /exec failed with exit code 1
pull / test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (ic3, portable, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (mv2, portable, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (resnet18, portable, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (resnet50, portable, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (mv3, portable, cmake, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (vit, portable, cmake, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-moshi-linux / linux-job (gh)
test_exported_decoder_xnnpack
pull / test-multimodal-linux (gemma3-4b) / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-parakeet-xnnpack-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-phi-3-mini-runner-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-quantized-aot-lib-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-selective-build-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-voxtral-realtime-xnnpack-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / test-vulkan-operators-linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / unittest / linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / unittest / windows / windows-job (gh)
Process completed with exit code 1.
pull / unittest-arm-backend-with-no-deps (test_pytest_models_tosa) / linux-job (gh)
RuntimeError: Command docker exec -t fda41404f4e28a7e25ed3f5ceb28bfc1ccda07de4d65ce6387d2d3d4da46ff8a /exec failed with exit code 1
pull / unittest-arm-backend-with-no-deps (test_pytest_ops_tosa) / linux-job (gh)
RuntimeError: Command docker exec -t c48e9ac744e5861cb2f6e3e1885ab28de4bf95169d62a494336953650dbf1f96 /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
AttributeError: '_ViewSpec' object has no attribute 'device_index'
pull / unittest-editable / windows / windows-job (gh)
Process completed with exit code 1.
pull / unittest-nxp-neutron / linux-job (gh)
RuntimeError: Command docker exec -t d98b9dfa150144b016364bcb61e1b427c32cba24ed2e6ac93d0c4a1f9350743a /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

Check Labels / Check labels (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…r device type Extends memory planning to separate device tensors from CPU tensors into distinct memory buffers. Non-CPU TensorSpecs (e.g., CUDA) are pre-assigned device-specific mem_ids before the greedy/naive algorithm runs, ensuring they get planned into independent memory buffers that never share space with CPU tensors. Differential Revision: [D97447105](https://our.internmc.facebook.com/intern/diff/D97447105/) ghstack-source-id: 355133801 Pull Request resolved: #18375

github-actions · 2026-03-20T18:22:45Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Gasoonjia requested review from JacobSzwejbka and larryliu0820 as code owners March 20, 2026 18:22

This was referenced Mar 20, 2026

[ET Device Support] Schema changes: device info on Tensor and buffer-level device array #17533

Open

[ET Device Support] TensorImpl carries device info #17534

Open

This was referenced Mar 20, 2026

[executorch] Propagate device metadata from partitioner result onto TensorSpecs #18078

Open

[ET Device Support] Propagate device info from TensorSpec into serialized Tensor #18079

Open

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 20, 2026

meta-codesync bot added fb-exported meta-exported labels Mar 20, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ET Device Support] Device-aware memory planning: separate buffers per device type#18375

[ET Device Support] Device-aware memory planning: separate buffers per device type#18375
Gasoonjia wants to merge 1 commit intogh/gasoonjia/145/basefrom
gh/gasoonjia/145/head

Gasoonjia commented Mar 20, 2026 •

edited

Loading

Uh oh!

pytorch-bot bot commented Mar 20, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Gasoonjia commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18375

❌ 62 New Failures, 1 Cancelled Job, 4 Pending

Uh oh!

github-actions bot commented Mar 20, 2026

This PR needs a release notes: label

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Gasoonjia commented Mar 20, 2026 •

edited

Loading

pytorch-bot bot commented Mar 20, 2026 •

edited

Loading

This PR needs a `release notes:` label