[ISSUE #9983] Prohibit CallerRunsPolicy in NettyRemotingServer to prevent blocking IO threads

[daguimu]

Problem

When a custom ExecutorService with CallerRunsPolicy is registered via registerProcessor or registerDefaultProcessor, and the thread pool becomes exhausted, the rejected task is executed on the caller thread — which is the Netty IO EventLoop thread. This blocks the IO thread from processing other requests/heartbeats, causing severe performance degradation or node unavailability.

Root Cause

No validation exists on the RejectedExecutionHandler of the provided executor in registerProcessor / registerDefaultProcessor. Users can silently misconfigure a ThreadPoolExecutor with CallerRunsPolicy, leading to IO thread blocking under high concurrency.

Fix

Added a rejectCallerRunsPolicy validation method that checks if the executor is a ThreadPoolExecutor with CallerRunsPolicy and throws IllegalArgumentException for fast-fail. The check is applied to:

• NettyRemotingServer#registerProcessor
• NettyRemotingServer#registerDefaultProcessor
• SubRemotingServer#registerProcessor
• SubRemotingServer#registerDefaultProcessor

Tests Added

• testRegisterProcessorRejectsCallerRunsPolicy — Verifies registerProcessor throws IllegalArgumentException when CallerRunsPolicy is used
• testRegisterDefaultProcessorRejectsCallerRunsPolicy — Verifies registerDefaultProcessor throws IllegalArgumentException when CallerRunsPolicy is used
• testRegisterProcessorAllowsNonCallerRunsPolicy — Verifies AbortPolicy (default) is accepted without error
• testRegisterProcessorAllowsNullExecutor — Verifies null executor (falls back to publicExecutor) works correctly
• testRegisterProcessorAllowsFixedThreadPool — Verifies standard FixedThreadPool is accepted

Impact

• Scope: Only affects registration-time validation, no runtime overhead
• Behavioral change: Executors with CallerRunsPolicy will now be rejected at registration time instead of silently causing IO thread blocking at runtime

Fixes #9983