Add debug logging option for rate-limited queries in QueryLogger

[gortiz]

Summary

This PR introduces a new mode for query logging when the broker query log rate limit is reached. The behavior is controlled by the broker.query.log.on.rate.limit configuration property, which now supports a new value: debug. The default remains drop.

Motivation

Previously, when the rate limit was reached, QueryLogger would drop query logs entirely. This made it difficult to capture and forward all query logs for further analysis or external processing (e.g., to separate files, syslog, or other appenders).

Changes

• Added support for a new value debug for the broker.query.log.on.rate.limit property.
• When set to debug, logs are emitted at the INFO level until the rate limit is reached, after which they are logged at the DEBUG level instead of being dropped.
• Any value other than debug retains the existing behavior (logs are dropped after the rate limit is reached).

Impact

• Backward compatible: the default remains drop, so existing deployments are unaffected unless the new mode is explicitly enabled.
• Provides more flexibility for operators to capture and route all query logs, even under high load.

[codecov-commenter]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
9374	2	9372	32

View the top 2 failed test(s) by shortest run time

org.apache.pinot.controller.api.PinotTableRestletResourceTest::testTableTasksCleanupWithNonActiveTasks

Stack Traces | 0.302s run time

org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 500 (Internal Server Error) with reason: "Failed to delete job: TaskQueue_SegmentGenerationAndPushTask_Task_SegmentGenerationAndPushTask_354bae43-85bf-41b3-a85c-79a6c0252742_1770660800311 from queue: TaskQueue_SegmentGenerationAndPushTask" while sending request: /tables/testTableTasksCleanup to controller: runnervmwffz4.bxfxhvyjb2je3bpitqmo1o3ped.bx.internal.cloudapp.net, version: Unknown

org.apache.pinot.controller.api.PinotTableRestletResourceTest::testTableTasksCleanupWithNonActiveTasks

Stack Traces | 0.317s run time

org.apache.pinot.common.exception.HttpErrorStatusException: Got error status code: 500 (Internal Server Error) with reason: "Failed to delete job: TaskQueue_SegmentGenerationAndPushTask_Task_SegmentGenerationAndPushTask_354bae43-85bf-41b3-a85c-79a6c0252742_1770660800311 from queue: TaskQueue_SegmentGenerationAndPushTask" while sending request: /tables/testTableTasksCleanup to controller: runnervmwffz4.bxfxhvyjb2je3bpitqmo1o3ped.bx.internal.cloudapp.net, version: Unknown

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}