Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 18 additions & 18 deletions examples/benchmark-react/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,34 +55,34 @@ Illustrative **relative** results with **baseline = 100%** (plain React useState

| Category | Scenarios (representative) | data-client | tanstack-query | swr | baseline |
|---|---|---:|---:|---:|---:|
| Navigation | `getlist-100`, `getlist-500`, `getlist-500-sorted` | ~96% | ~98% | ~99% | **100%** |
| Navigation | `list-detail-switch-10` | **~949%** | ~199% | ~203% | 100% |
| Mutations | `update-entity`, `update-user`, `update-entity-sorted`, `update-entity-multi-view`, `unshift-item`, `delete-item`, `move-item` | **~4486%** | ~96% | ~99% | 100% |
| Scaling (10k items) | `update-user-10000` | **~2006%** | ~84% | ~103% | 100% |
| Navigation | `getlist-100`, `getlist-500`, `getlist-500-sorted` | ~95% | ~97% | ~99% | **100%** |
| Navigation | `list-detail-switch-10` | **~851%** | ~233% | ~247% | 100% |
| Mutations | `update-entity`, `update-user`, `update-entity-sorted`, `update-entity-multi-view`, `unshift-item`, `delete-item`, `move-item` | **~4442%** | ~97% | ~99% | 100% |
| Scaling (10k items) | `update-user-10000` | **~6408%** | ~94% | ~100% | 100% |


## Latest measured results (network simulation on)

Median ops/s per scenario; range is approximate 95% CI margin from the runner (`stats.ts`). **Network simulation** applies the per-RPC delays in `bench/scenarios.ts` (`NETWORK_SIM_DELAYS`, e.g. `fetchIssueList` 80 ms, `updateUser` 50 ms) so list refetches after an author update pay extra latency compared to normalized propagation.
Median ops/s per scenario; range is approximate 95% CI margin from the runner (`stats.ts`). **Network simulation** uses response-size-based delays (`NETWORK_SIM_CONFIG` in `bench/scenarios.ts`: 40 ms base latency + 1 ms per 20 records) so list refetches after an author update pay extra latency compared to normalized propagation.

Run: **2026-03-21**, Linux (WSL2), `yarn build:benchmark-react`, static preview + `env -u CI npx tsx bench/runner.ts --network-sim true` (all libraries; memory scenarios not included). Numbers are **machine-specific**; use them for relative comparison between libraries, not as absolutes.
Run: **2026-03-22**, Linux (WSL2), `yarn build:benchmark-react`, static preview + `env -u CI npx tsx bench/runner.ts --network-sim true` (all libraries; memory scenarios not included). Numbers are **machine-specific**; use them for relative comparison between libraries, not as absolutes.

| Scenario | data-client | tanstack-query | swr | baseline |
|---|---:|---:|---:|---:|
| **Navigation** | | | | |
| `getlist-100` | 11.20 ± 0.03 | 11.27 ± 0.02 | 11.43 ± 0.07 | 11.55 ± 0.02 |
| `getlist-500` | 9.78 ± 0.12 | 10.01 ± 0.13 | 10.16 ± 0.13 | 10.22 ± 0.07 |
| `getlist-500-sorted` | 9.82 ± 0.16 | 10.08 ± 0.13 | 10.21 ± 0.07 | 10.29 ± 0.06 |
| `list-detail-switch-10` | 6.93 ± 1.02 | 1.45 ± 0.04 | 1.48 ± 0.08 | 0.73 ± 0.00 |
| `getlist-100` | 18.48 ± 0.02 | 18.62 ± 0.07 | 19.12 ± 0.02 | 19.34 ± 0.09 |
| `getlist-500` | 11.45 ± 0.21 | 11.92 ± 0.18 | 11.96 ± 0.04 | 12.06 ± 0.08 |
| `getlist-500-sorted` | 11.48 ± 0.39 | 11.81 ± 0.22 | 12.00 ± 0.34 | 12.08 ± 0.37 |
| `list-detail-switch-10` | 6.13 ± 0.74 | 1.68 ± 0.07 | 1.78 ± 0.12 | 0.72 ± 0.00 |
| **Mutations** | | | | |
| `update-entity` | 357.14 ± 11.48 | 7.01 ± 0.02 | 7.02 ± 0.02 | 7.22 ± 0.00 |
| `update-user` | 333.33 ± 14.44 | 7.01 ± 0.02 | 7.17 ± 0.03 | 7.22 ± 0.01 |
| `update-entity-sorted` | 312.50 ± 23.44 | 7.08 ± 0.00 | 7.07 ± 0.03 | 7.28 ± 0.01 |
| `update-entity-multi-view` | 357.14 ± 52.30 | 6.82 ± 0.34 | 6.88 ± 0.39 | 7.14 ± 0.36 |
| `update-user-10000` | 97.09 ± 7.73 | 4.07 ± 0.02 | 4.97 ± 0.02 | 4.84 ± 0.03 |
| `unshift-item` | 285.71 ± 4.90 | 6.92 ± 0.02 | 7.16 ± 0.00 | 7.16 ± 0.02 |
| `delete-item` | 312.50 ± 9.77 | 6.93 ± 0.01 | 7.15 ± 0.01 | 7.16 ± 0.01 |
| `move-item` | 285.71 ± 10.61 | 6.39 ± 0.02 | 6.83 ± 0.00 | 6.82 ± 0.00 |
| `update-entity` | 333.33 ± 4.22 | 6.95 ± 0.00 | 6.94 ± 0.02 | 7.17 ± 0.00 |
| `update-user` | 322.58 ± 11.79 | 6.97 ± 0.01 | 7.15 ± 0.00 | 7.15 ± 0.02 |
| `update-entity-sorted` | 285.71 ± 30.41 | 7.04 ± 0.01 | 7.05 ± 0.02 | 7.23 ± 0.01 |
| `update-entity-multi-view` | 344.83 ± 16.69 | 5.89 ± 0.77 | 5.89 ± 0.82 | 5.97 ± 0.05 |
| `update-user-10000` | 98.04 ± 5.79 | 1.44 ± 0.01 | 1.53 ± 0.00 | 1.53 ± 0.01 |
| `unshift-item` | 285.71 ± 11.11 | 6.89 ± 0.02 | 7.11 ± 0.01 | 7.11 ± 0.01 |
| `delete-item` | 312.50 ± 14.76 | 6.87 ± 0.01 | 7.09 ± 0.01 | 7.10 ± 0.00 |
| `move-item` | 256.41 ± 8.77 | 6.34 ± 0.06 | 6.80 ± 0.01 | 6.77 ± 0.01 |

[Measured on a Ryzen 9 7950X; 64 GB RAM; Ubuntu (WSL2); Node 24.12.0; Chromium (Playwright)]

Expand Down
10 changes: 6 additions & 4 deletions examples/benchmark-react/bench/runner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
LIBRARIES,
RUN_CONFIG,
ACTION_GROUPS,
NETWORK_SIM_DELAYS,
NETWORK_SIM_CONFIG,
} from './scenarios.js';
import { computeStats, isConverged } from './stats.js';
import { parseTraceDuration } from './tracing.js';
Expand Down Expand Up @@ -187,8 +187,9 @@ async function runScenario(

if (networkSim) {
await (bench as any).evaluate(
(api: any, delays: Record<string, number>) => api.setMethodDelays(delays),
NETWORK_SIM_DELAYS,
(api: any, config: { baseLatencyMs: number; recordsPerMs: number }) =>
api.setNetworkSim(config),
NETWORK_SIM_CONFIG,
);
}

Expand Down Expand Up @@ -257,8 +258,9 @@ async function runScenario(
(api: any, [action, n]: [string, number]) => api[action](n),
[preMountAction, mountCount],
);
const preMountTimeout = networkSim ? 60000 : 10000;
await page.waitForSelector('[data-bench-complete]', {
timeout: 10000,
timeout: preMountTimeout,
state: 'attached',
});
const preMountTimedOut = await harness.evaluate(el =>
Expand Down
15 changes: 5 additions & 10 deletions examples/benchmark-react/bench/scenarios.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,10 @@
import type { BenchAPI, Scenario, ScenarioSize } from '../src/shared/types.js';

/** Per-method network latency used when --network-sim is enabled (default: on). */
export const NETWORK_SIM_DELAYS: Record<string, number> = {
fetchIssueList: 80,
fetchIssue: 50,
fetchUser: 50,
createIssue: 50,
updateIssue: 50,
updateUser: 50,
deleteIssue: 50,
deleteUser: 50,
/** Response-size-based network simulation used when --network-sim is enabled (default: on).
* Delay per request = baseLatencyMs + ceil(recordCount / recordsPerMs). */
export const NETWORK_SIM_CONFIG = {
baseLatencyMs: 40,
recordsPerMs: 20,
};

export interface RunProfile {
Expand Down
4 changes: 2 additions & 2 deletions examples/benchmark-react/src/shared/benchHarness.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import { captureSnapshot, getReport } from './refStability';
import {
flushPendingMutations,
seedIssueList,
setMethodDelays,
setNetworkDelay,
setNetworkSim,
} from './server';
import type { BenchAPI } from './types';

Expand Down Expand Up @@ -337,7 +337,7 @@ export function useBenchState() {
captureRefSnapshot,
getRefStabilityReport,
setNetworkDelay,
setMethodDelays,
setNetworkSim,
flushPendingMutations,
setRenderLimit,
...libraryActions,
Expand Down
8 changes: 5 additions & 3 deletions examples/benchmark-react/src/shared/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,10 +136,12 @@ export function setNetworkDelay(ms: number): void {
});
}

export function setMethodDelays(delays: Record<string, number>): void {
export function setNetworkSim(
config: { baseLatencyMs: number; recordsPerMs: number } | null,
): void {
worker.postMessage({
id: nextId++,
method: 'setMethodDelays',
params: { delays },
method: 'setNetworkSim',
params: { config },
});
}
21 changes: 15 additions & 6 deletions examples/benchmark-react/src/shared/server.worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,16 @@ declare const self: DedicatedWorkerGlobalScope;
// ── NETWORK DELAY ────────────────────────────────────────────────────────

let networkDelayMs = 0;
let methodDelays: Record<string, number> = {};
let networkSim: { baseLatencyMs: number; recordsPerMs: number } | null = null;

function respond(id: number, method: string, value: unknown) {
function respond(id: number, _method: string, value: unknown) {
const json = JSON.stringify(value);
const delay = methodDelays[method] ?? networkDelayMs;
const recordCount = Array.isArray(value) ? value.length : 1;
const delay =
networkSim ?
networkSim.baseLatencyMs +
Math.ceil(recordCount / networkSim.recordsPerMs)
: networkDelayMs;
if (delay <= 0) {
self.postMessage({ id, result: json });
} else {
Expand Down Expand Up @@ -250,10 +255,14 @@ const methods: Record<string, (params: any) => unknown> = {
seedIssueList: ({ issues }: { issues: Issue[] }) => seedIssueList(issues),
setNetworkDelay: ({ ms }: { ms: number }) => {
networkDelayMs = ms;
methodDelays = {};
networkSim = null;
},
setMethodDelays: ({ delays }: { delays: Record<string, number> }) => {
methodDelays = delays;
setNetworkSim: ({
config,
}: {
config: { baseLatencyMs: number; recordsPerMs: number } | null;
}) => {
networkSim = config;
},
};

Expand Down
8 changes: 5 additions & 3 deletions examples/benchmark-react/src/shared/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,12 @@ export interface BenchAPI {
init(count: number): void;
updateEntity(id: number): void;
updateUser(login: string): void;
/** Set simulated per-request network latency (ms). 0 disables and clears per-method delays. */
/** Set simulated per-request network latency (ms). 0 disables and clears network sim. */
setNetworkDelay(ms: number): void;
/** Set per-method network latency overrides (e.g. { fetchIssueList: 80, fetchIssue: 50 }). */
setMethodDelays(delays: Record<string, number>): void;
/** Enable/disable response-size-based network simulation. Delay = baseLatencyMs + ceil(recordCount / recordsPerMs). Pass null to disable. */
setNetworkSim(
config: { baseLatencyMs: number; recordsPerMs: number } | null,
): void;
/** Wait for all deferred server mutations to settle before next iteration. */
flushPendingMutations(): Promise<void>;
unmountAll(): void;
Expand Down
Loading