Skip to content

[ISSUE #9926] Optimize memory allocation in parsePublishMessageQueues#10206

Open
daguimu wants to merge 1 commit intoapache:developfrom
daguimu:fix/optimize-parsePublishMessageQueues-9926
Open

[ISSUE #9926] Optimize memory allocation in parsePublishMessageQueues#10206
daguimu wants to merge 1 commit intoapache:developfrom
daguimu:fix/optimize-parsePublishMessageQueues-9926

Conversation

@daguimu
Copy link
Copy Markdown

@daguimu daguimu commented Mar 25, 2026

Problem

Under high throughput workloads (e.g., 50K LMQ tps), parsePublishMessageQueues causes massive memory allocation due to:

  1. Creating a new ArrayList (with default capacity 10) for every call, even when namespace is empty
  2. Repeated getNamespace() calls per queue element
  3. Unnecessary ArrayList growth/copy when the list size exceeds initial capacity

Profiling by the issue reporter showed this method as a significant source of memory allocation overhead.

Root Cause

The method unconditionally creates a new list and copies all queues with namespace stripping, even when no namespace is configured (the most common deployment scenario). The ArrayList is also created with default capacity instead of being pre-sized.

Fix

  1. Early return when namespace is empty: Return the original messageQueueList directly, avoiding all allocation
  2. Pre-allocate ArrayList: Use new ArrayList<>(messageQueueList.size()) to avoid internal array resizing
  3. Cache namespace: Extract getNamespace() call outside the loop to avoid repeated method invocations

Tests Added

  • testParsePublishMessageQueuesReturnsOriginalListWhenNamespaceEmpty — Verifies same list instance is returned when namespace is empty string
  • testParsePublishMessageQueuesReturnsOriginalListWhenNamespaceNull — Verifies same list instance is returned when namespace is null
  • testParsePublishMessageQueuesStripsNamespace — Verifies namespace is correctly stripped when present
  • testParsePublishMessageQueuesPreservesOrderAndSize — Verifies correct behavior with 100 queues

Impact

  • Zero allocation in the common case (no namespace configured)
  • No behavioral change when namespace is configured, only reduced memory allocation
  • Hot path optimization for high-throughput producers

Fixes #9926

…Queues

Skip list creation entirely when namespace is empty (most common case),
and pre-allocate ArrayList with known capacity when namespace stripping
is needed. Also cache the namespace value to avoid repeated method calls.

Fixes apache#9926
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] massive memory allocation in the parsePublishMessageQueues

1 participant