Simplified benchmark runner by franz1981 · Pull Request #97 · franz1981/Netty-VirtualThread-Scheduler

franz1981 · 2026-03-23T08:12:46Z

No description provided.

…anualIoEventLoopGroup

…Y_SCHEDULER), add mockless mode, and simplify benchmark runner configuration

…ss+pidstat crash, add BACKLOG - README: replace toy examples with practical ones showing lscpu topology analysis, perf stat interpretation, and NUMA-aware cpuset allocation - Fix MOCK_PID unbound variable when running mockless mode with pidstat enabled - Add BACKLOG.md with investigation findings on kernel CPU accounting gap

franz1981 · 2026-03-23T10:48:02Z

@copilot scrutinize If the mockless behavior is correct and same while running with a netty scheduler vs virtual netty modes

Copilot · 2026-03-23T10:48:08Z

@franz1981 I've opened a new pull request, #99, to work on those changes. Once the pull request is ready, I'll request review from you.

franz1981 · 2026-03-23T10:49:15Z

@copilot nope just work on this PR and suggest changes in case you think there is an issue instead

Copilot · 2026-03-23T10:49:20Z

@franz1981 I've opened a new pull request, #100, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot

Pull request overview

This PR simplifies the benchmark runner configuration by introducing a consolidated “mode” concept, updating the runner script/README to prefer CLI flags, and adding a new Netty “virtual IO loop” implementation used by the benchmarks.

Changes:

Add VirtualMultithreadManualIoEventLoopGroup + ManualIoEventLoopTask to run Netty IO event loops on virtual threads.
Simplify benchmark runner modes (NON_VIRTUAL_NETTY, REACTIVE, VIRTUAL_NETTY, NETTY_SCHEDULER) and add a --mockless fast-path.
Refactor run-benchmark.sh + README toward CLI-first configuration and updated examples.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
core/src/main/java/io/netty/loom/VirtualMultithreadManualIoEventLoopGroup.java	New event loop group that spawns a virtual thread per manual IO event loop task.
core/src/main/java/io/netty/loom/ManualIoEventLoopTask.java	New runnable `ManualIoEventLoop` wrapper that drives IO/task execution in a loop.
benchmark-runner/src/test/java/io/netty/loom/benchmark/runner/BenchmarkIntegrationTest.java	Narrows/adjusts integration test matrix to the new `Mode`-based server configuration.
benchmark-runner/src/main/java/io/netty/loom/benchmark/runner/ReactiveHandoffHandler.java	Simplifies reactive handler configuration and removes timeout toggles.
benchmark-runner/src/main/java/io/netty/loom/benchmark/runner/HandoffHttpServer.java	Introduces `Mode` + `mockless` support and rewires server setup accordingly.
benchmark-runner/scripts/run-benchmark.sh	CLI-flag parsing, cpuset variable renames, mode-based JVM args, and mockless support.
benchmark-runner/README.md	Documents CLI-first usage, new modes, mockless mode, and updated benchmarking guidance.
BACKLOG.md	Adds a research/ideas backlog document for benchmarking/accounting observations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-23T10:51:27Z

benchmark-runner/src/main/java/io/netty/loom/benchmark/runner/HandoffHttpServer.java

 			if (uri.equals("/") || uri.startsWith("/fruits")) {
-				// Hand off to virtual thread for blocking processing
-				orderedExecutorService.execute(() -> {
-					doBlockingProcessing(ctx, eventLoop, keepAlive);
-				});
+				// Hand off to virtual thread for processing
+				if (mockless) {
+					orderedExecutorService.execute(() -> {
+						doMocklessProcessing(ctx, eventLoop, keepAlive);
+					});
+				} else {


mockless introduces a new execution path (skipping the HTTP call and using CACHED_JSON_BYTES) but there’s no integration test exercising --mockless (including error handling and keep-alive behavior). Consider adding a parameterized test configuration for mockless=true to ensure this mode keeps working.

Copilot · 2026-03-23T10:51:27Z

core/src/main/java/io/netty/loom/ManualIoEventLoopTask.java

+	private static final long RUNNING_YIELD_US = TimeUnit.MICROSECONDS
+			.toNanos(Integer.getInteger("io.netty.loom.running.yield.us", 1));


RUNNING_YIELD_US is computed in nanoseconds (TimeUnit.MICROSECONDS.toNanos(...)), so the name/unit is misleading and makes the subsequent run(..., RUNNING_YIELD_US) calls hard to reason about. Rename the constant (or store the raw microseconds and convert at call sites) so the unit matches the identifier.

Copilot · 2026-03-23T10:51:27Z

benchmark-runner/src/main/java/io/netty/loom/benchmark/runner/HandoffHttpServer.java

+				  --threads <n>                  Number of event loop threads (default: 1)
+				  --io <epoll|nio|io_uring>      I/O type (default: epoll)
+				  --mockless                     Skip mock server; do Jackson work inline (default: off)
+				  --mode <mode>                  Server mode (default: virtual_thread)


The help text says --mode <mode> defaults to virtual_thread, but main() initializes mode = Mode.NON_VIRTUAL_NETTY. This is user-visible and will cause confusion when running the benchmark runner. Update the printed default (and any mode naming) to match the actual default behavior.

Suggested change

--mode <mode> Server mode (default: virtual_thread)

--mode <mode> Server mode (default: NON_VIRTUAL_NETTY)

benchmark-runner/src/main/java/io/netty/loom/benchmark/runner/ReactiveHandoffHandler.java

Copilot · 2026-03-23T10:51:28Z

benchmark-runner/scripts/run-benchmark.sh

 SERVER_FJ_PARALLELISM="${SERVER_FJ_PARALLELISM:-}"  # ForkJoinPool parallelism (empty = JVM default)
-SERVER_NO_TIMEOUT="${SERVER_NO_TIMEOUT:-false}"  # Disable HTTP client timeout
-SERVER_REACTIVE="${SERVER_REACTIVE:-false}"  # Use reactive handler with Project Reactor
+SERVER_MODE="${SERVER_MODE:-NON_VIRTUAL_NETTY}"  # Server mode: NON_VIRTUAL_NETTY, REACTIVE, VIRTUAL_NETTY


The comment describing supported SERVER_MODE values omits NETTY_SCHEDULER, but the script explicitly supports it (and the help text later lists it). Update the comment to reflect the actual supported modes to avoid configuration mistakes.

Suggested change

SERVER_MODE="${SERVER_MODE:-NON_VIRTUAL_NETTY}" # Server mode: NON_VIRTUAL_NETTY, REACTIVE, VIRTUAL_NETTY

SERVER_MODE="${SERVER_MODE:-NON_VIRTUAL_NETTY}" # Server mode: NON_VIRTUAL_NETTY, REACTIVE, VIRTUAL_NETTY, NETTY_SCHEDULER

Copilot · 2026-03-23T10:51:28Z

benchmark-runner/src/main/java/io/netty/loom/benchmark/runner/HandoffHttpServer.java

+					ByteBuf content = Unpooled.copiedBuffer("{\"error\":\"" + e.getMessage() + "\"}",
+							CharsetUtil.UTF_8);
+					FullHttpResponse response = new DefaultFullHttpResponse(HttpVersion.HTTP_1_1,
+							HttpResponseStatus.INTERNAL_SERVER_ERROR, content);
+					response.headers().set(HttpHeaderNames.CONTENT_TYPE, HttpHeaderValues.APPLICATION_JSON);
+					response.headers().setInt(HttpHeaderNames.CONTENT_LENGTH, content.readableBytes());
+					ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE);
+				});


In doMocklessProcessing the error path builds a JSON string via concatenation using e.getMessage(). If the message contains quotes, backslashes, or newlines, the response becomes invalid JSON and may leak internal details. Prefer serializing a structured error payload via Jackson (or at least escaping the message) and consider honoring keepAlive/not always closing the connection on errors.

franz1981 added 4 commits March 23, 2026 09:15

Add VIRTUAL_NETTY support: ManualIoEventLoopTask, VirtualMultithreadM…

aecb1f4

…anualIoEventLoopGroup

Introduce Mode enum (NON_VIRTUAL_NETTY, REACTIVE, VIRTUAL_NETTY, NETT…

46faf55

…Y_SCHEDULER), add mockless mode, and simplify benchmark runner configuration

Add CLI flags to run-benchmark.sh and rename *_TASKSET to *_CPUSET

b96e088

Remove FJ affinity option from benchmark runner

5017502

franz1981 force-pushed the mode_simplified branch from c434b52 to 5017502 Compare March 23, 2026 08:22

franz1981 added 2 commits March 23, 2026 10:09

Fix config banner to show effective pollerMode for NETTY_SCHEDULER

ab2fafd

franz1981 requested a review from Copilot March 23, 2026 10:47

Copilot started reviewing on behalf of franz1981 March 23, 2026 10:47 View session

Copilot AI mentioned this pull request Mar 23, 2026

Scrutiny: mockless mode correctness across NETTY_SCHEDULER and VIRTUAL_NETTY #99

Draft

Copilot AI mentioned this pull request Mar 23, 2026

Expand integration test coverage: NON_VIRTUAL_NETTY and mockless modes #100

Draft

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Simplified benchmark runner#97

Simplified benchmark runner#97
franz1981 wants to merge 6 commits intomasterfrom
mode_simplified

franz1981 commented Mar 23, 2026

Uh oh!

franz1981 commented Mar 23, 2026

Uh oh!

Copilot AI commented Mar 23, 2026

Uh oh!

franz1981 commented Mar 23, 2026

Uh oh!

Copilot AI commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		private static final long RUNNING_YIELD_US = TimeUnit.MICROSECONDS
		.toNanos(Integer.getInteger("io.netty.loom.running.yield.us", 1));

	--mode <mode> Server mode (default: virtual_thread)
	--mode <mode> Server mode (default: NON_VIRTUAL_NETTY)

	SERVER_MODE="${SERVER_MODE:-NON_VIRTUAL_NETTY}" # Server mode: NON_VIRTUAL_NETTY, REACTIVE, VIRTUAL_NETTY
	SERVER_MODE="${SERVER_MODE:-NON_VIRTUAL_NETTY}" # Server mode: NON_VIRTUAL_NETTY, REACTIVE, VIRTUAL_NETTY, NETTY_SCHEDULER

Conversation

franz1981 commented Mar 23, 2026

Uh oh!

franz1981 commented Mar 23, 2026

Uh oh!

Copilot AI commented Mar 23, 2026

Uh oh!

franz1981 commented Mar 23, 2026

Uh oh!

Copilot AI commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants