Vulkan: export allocator for custom contexts by xFile3160 · Pull Request #8871 · halide/Halide

xFile3160 · 2025-11-17T21:55:24Z

This is for applications that already own a Vulkan context and want Halide to
run work on that same context.

The case I care about is an application with its own VkInstance, VkDevice,
and queue. The application passes those handles to Halide from the Vulkan
context callback. Halide should not create or destroy that context, but it still
needs its own runtime allocator for shader modules, staging buffers, descriptor
resources, and other internal Vulkan objects.

This PR exposes acquire/release helpers for Halide's opaque Vulkan allocator.
The intended flow is:

The application owns the Vulkan context.
The application asks Halide to create or validate an allocator for that
context.
The application stores that allocator next to its Vulkan context state.
Later halide_vulkan_acquire_context calls return the same allocator and
Vulkan handles back to Halide.
When the external context is torn down, the application releases the Halide
allocator before destroying its VkDevice.

The release helper only tears down Halide-owned allocator and shader-module
state. It does not destroy the caller's Vulkan instance, device, queue, or
debug messenger.

This also adds setter functions for the Vulkan acquire/release context
callbacks. The weak-symbol override path still works, but setters are simpler
when Halide is linked into a larger library or used in an environment where
symbol interposition is not reliable.

The Vulkan compilation cache is keyed by allocator instead of raw VkDevice,
so cache cleanup follows the same lifetime as the allocator being released.

The acquire/release AOT test now covers the Vulkan path as well. It creates real
Vulkan handles, installs the callback setters, runs Halide through the callback
path with an externally stored allocator, and releases that allocator before
restoring the default context handling.

xFile3160 · 2025-11-17T21:56:48Z

Full discussion available here: #8715 (comment)

alexreinking · 2025-11-18T16:51:19Z

@derek-gerstmann could you review this PR? Seems you were discussing the related issue.

xFile3160 · 2026-01-20T21:42:35Z

Any news about this? Should I rebase?

alexreinking · 2026-01-20T23:14:32Z

Hi @xFile3160 -- happy new year! Yes, please rebase (looks like you just did). I'll ping @derek-gerstmann again to look at this since he has in-depth knowledge of the Vulkan backend.

derek-gerstmann · 2026-01-20T23:27:22Z

Thanks for the reminder! I'll look this over this week!

derek-gerstmann · 2026-01-20T23:49:21Z

+//   with the same locking used by the custom acquire/release implementations. This allows the allocator to be
+//   saved for future halide_vulkan_acquire_context calls that Halide will automatically issue to retrieve
+//   the custom context.
+extern int halide_vulkan_export_memory_allocator(void *user_context,


I don't understand the need for this method, or for the corresponding release method. The allocator should be stored in your custom context, and held onto for the lifetime of the context. The context manages lifespan of the allocator.

derek-gerstmann · 2026-01-20T23:49:32Z

+// - halide_vulkan_memory_allocator_release
+//   releases the internally allocated memory allocator, important for proper memory cleanup. Must have overridden halide_vulkan_acquire_context
+//   and halide_vulkan_release_context, and must coordinate with the same locking as the custom implementations.
+extern int halide_vulkan_memory_allocator_release(void *user_context,


See above comment.

derek-gerstmann · 2026-01-20T23:50:38Z

    return is_initialized;
 }

+WEAK int halide_vulkan_export_memory_allocator(void *user_context, halide_vulkan_memory_allocator *allocator) {


This doesn't actually do anything other than check to see if the allocator is null.

derek-gerstmann · 2026-01-20T23:52:00Z

    return destroy_status;
 }

+WEAK int halide_vulkan_memory_allocator_release(void *user_context,


Not sure I understand the intent ... was it to have a public method to invoke the destructor for the allocator?

derek-gerstmann · 2026-01-20T23:57:12Z

            error = halide_error_code_device_interface_no_device;
            halide_error_no_device_interface(user_context);
        }
+        // If user overrode halide_vulkan_acquire_context and returned nullptr for allocator,


This class shouldn't be doing anything other than holding a lock on the context. It's just a convenient wrapper for the internal methods to have a lock that lives within a scope.

derek-gerstmann · 2026-04-22T20:51:17Z

+//   with the same locking used by the custom acquire/release implementations. This allows the allocator to be
+//   saved for future halide_vulkan_acquire_context calls that Halide will automatically issue to retrieve
+//   the custom context.
+extern int halide_vulkan_export_memory_allocator(void *user_context,


I'd suggest following the conventions of the context methods and naming this halide_vulkan_acquire_memory_allocator.

derek-gerstmann · 2026-04-22T20:51:39Z

+// - halide_vulkan_memory_allocator_release
+//   releases the internally allocated memory allocator, important for proper memory cleanup. Must have overridden halide_vulkan_acquire_context
+//   and halide_vulkan_release_context, and must coordinate with the same locking as the custom implementations.
+extern int halide_vulkan_memory_allocator_release(void *user_context,


Same as above. I'd suggest I'd suggest naming this halide_vulkan_release_memory_allocator.

derek-gerstmann · 2026-04-22T20:52:36Z

 }

+WEAK int halide_vulkan_export_memory_allocator(void *user_context, halide_vulkan_memory_allocator *allocator) {
+    halide_mutex_lock(&thread_lock);


This default implementation doesn't actually do anything ... shouldn't it return the allocator associated with the context?

derek-gerstmann · 2026-04-22T20:54:55Z

+        return halide_error_code_buffer_argument_is_null;
+    }
+
+    return vk_release_memory_allocator(user_context, (VulkanMemoryAllocator *)allocator,


Lifetime management is an issue here. How do we know there are no remaining uses for the allocator? Also, allocators are specific to the context, so we need to make sure the given allocator matches the one associated with the given context.

derek-gerstmann · 2026-04-22T20:57:11Z

+            halide_start_clock(user_context);
+#endif
+            // make sure halide vulkan is loaded BEFORE creating allocator
+            debug(user_context) << "VulkanContext: Loading Vulkan function pointers for context override...\n";


This is not the right place to initialize device function pointers. They are specific to the context, and should only be initialized once, which is why they are only done in the acquire_context method.

xFile3160 · 2026-04-22T21:19:04Z

I've changed this patch quiet a bit actually. But @derek-gerstmann your comments make absolutely sense, and I'm going to address/explain the intent and the new API a bit better soon. Sorry, I haven't followed up either updating this patch.

When an application provides its own Vulkan context, Halide still needs its runtime allocator for shader modules, staging buffers, and internal Vulkan resources. Add public acquire/release helpers for the opaque Vulkan allocator. The acquire helper creates or validates an allocator for the supplied instance/device pair. The release helper tears down Halide-owned allocator state without destroying the caller's Vulkan instance, device, or queue. Key the Vulkan compilation cache by allocator instead of VkDevice so cache cleanup follows the allocator lifetime.

Weak symbol overrides are not reliable in every embedding environment. Add setter functions for the Vulkan acquire/release context callbacks, while keeping the existing weak symbols as the public runtime entry points. Passing null restores Halide's default context management.

alexreinking requested a review from halidebuildbots November 17, 2025 22:31

xFile3160 force-pushed the main branch from b0258f8 to 5b36964 Compare January 20, 2026 21:56

xFile3160 closed this Feb 16, 2026

xFile3160 force-pushed the main branch from f802490 to 1877f41 Compare February 16, 2026 15:49

xFile3160 reopened this Feb 16, 2026

alexreinking requested a review from derek-gerstmann February 16, 2026 16:36

derek-gerstmann requested changes Apr 22, 2026

View reviewed changes

xFile3160 force-pushed the main branch from 5bb201a to eaa2054 Compare April 25, 2026 08:40

xFile3160 mentioned this pull request Apr 25, 2026

Vulkan: wrap external buffers as regions #9110

Open

xFile3160 marked this pull request as draft April 28, 2026 08:00

xFile3160 added 2 commits April 28, 2026 12:25

xFile3160 force-pushed the main branch from 72360f6 to 396421a Compare April 28, 2026 19:27

Vulkan: test external context callbacks

4b60b83

xFile3160 marked this pull request as ready for review April 28, 2026 20:28

xFile3160 requested a review from derek-gerstmann April 28, 2026 20:29

Conversation

xFile3160 commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xFile3160 commented Nov 17, 2025

Uh oh!

alexreinking commented Nov 18, 2025

Uh oh!

xFile3160 commented Jan 20, 2026

Uh oh!

alexreinking commented Jan 20, 2026

Uh oh!

derek-gerstmann commented Jan 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xFile3160 commented Apr 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

xFile3160 commented Nov 17, 2025 •

edited

Loading