parallel_scheduler for P2079 Section 4.1 User Facing API (ref #6601)

[charan-003]

Fixes #6601

Implement P2079R7 parallel_scheduler in HPX Core

This issue tracks the implementation of hpx::execution::experimental::parallel_scheduler in HPX core, aligning with P2079R7 (Parallel scheduler, Section 4.1) by wrapping thread_pool_policy_scheduler with hpx::launch policies. The goal is to provide a standards-compliant parallel scheduler with global access, cancellation support, task chaining, and bulk operations.

Implementation Checklist

• Phase 1: Core Implementation

• Implement hpx::execution::experimental::parallel_scheduler class per P2079R7 Section 4.1.

• Add schedule() returning parallel_sender with sender_concept and completion_signatures:

• Supports set_value_t(), set_stopped_t(), set_error_t(std::exception_ptr).

• Wrap thread_pool_policy_scheduler with hpx::launch (async/sync policies).

• Implement noexcept move/copy constructors and assignment operators.

• Add operator== returning true for scheduler comparison.

• Return forward_progress_guarantee::parallel via get_forward_progress_guarantee_t.

• Implement then operation for task chaining with parallel_sender.

• Code to rely on STDEXEC.

• Phase 2: Bulk Support

• Extend bulk_t customization to delegate to thread_pool_scheduler_bulk.

• Implement bulk_sender for parallel bulk execution per P2079R7.

• Support bulk_chunked and bulk_unchunked operations.

• Phase 3: Replaceability API

• Implement std::execution::system_context_replaceability namespace per P2079R7 Section 4.2.

• Add query_parallel_scheduler_backend() returning shared_ptr<parallel_scheduler>.

• Support link-time replaceability using weak symbols.

• Implement receiver and bulk_item_receiver interfaces for frontend-backend interaction.

• Ensure storage handling for scheduling operations.

[StellarBot]

Can one of the admins verify this patch?

[codacy-production]

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ +0.45%	✅ 94.84%

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (6e3a98b)	193599	164650	85.05%
Head commit (1267582)	214837 (+21238)	183682 (+19032)	85.50% (+0.45%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#6655)	155	147	94.84%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

[charan-003]

clang format

[charan-003]

Update:

• Moved parallel_scheduler, parallel_sender, and bulk_sender to a new header: hpx/executors/parallel_scheduler.hpp
• Removed related code from thread_pool_scheduler.hpp

@hkaiser — ready for re-review when convenient. Thanks for the guidance!

[charan-003]

@hkaiser Can you please verify it now? I'm not sure why those 2 tests are failing.

[hkaiser]

Wow, I'm impressed! I have a couple of minor comments, though.

[hkaiser]

Also, you may want to move all types that are don't have to visible by users into namespace detail to avoid name clashes with other code.

[charan-003]

Also, you may want to move all types that are don't have to visible by users into namespace detail to avoid name clashes with other code.

Sure, I'll do that soon
Thank you so much

[charan-003]

i'm not sure why it's failing for clang_format. i used clang-format -i ` to make sure it's in the format

[charan-003]

@hkaiser
The parallel_bulk_operation_state implements a custom chunking strategy for bulk operations. It divides the workload into chunks based on the number of threads in the underlying thread_pool_scheduler’s pool (get_os_thread_count()), then schedules each chunk as a separate task using exec.execute()

The custom bulk chunking might duplicate functionality in thread_pool_scheduler_bulk.hpp

Offers a parallel execution option that leverages HPX’s thread pool, maintaining high performance for CPU-bound tasks.

While this version lacks P2079R7’s replaceability API (section 4.2) and many more, it could be extended to support it ( query_parallel_scheduler_backend()). This would let HPX users swap in custom schedulers (integrating with GPUs or other runtimes like oneTBB), enhancing HPX’s flexibility and composability

will try to work on this :))

[codacy-production]

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ -0.30%	✅ 88.81%

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (15afc8b)	244963	210174	85.80%
Head commit (7025dfd)	219167 (-25796)	187374 (-22800)	85.49% (-0.30%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#6655)	268	238	88.81%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

[hkaiser]

i'm not sure why it's failing for clang_format. i used clang-format -i ` to make sure it's in the format

The CI uses an older version of clang-format that does not always agree with newer versions on how to format things. Simply surround the offending lines with

// clang-format off
    ....
// clang-format on

to disable it checking those.

[charan-003]

@hkaiser @isidorostsa
This version is about adding P2079R7 parallel_scheduler to HPX core

Implement parallel_scheduler wrapping thread_pool_policy_schedulerhpx::launch, aligning with P2079R7 user-facing API:

• Add get_parallel_scheduler() for global access with async/sync policy selection.
• Include parallel_sender with set_value_t(), set_stopped_t(), set_error_t(std::exception_ptr).
• Support cancellation via stop token in operation_state::start_t.
• Ensure parallel forward progress guarantee.
• Add then operation for task chaining.
• Add tests for progress, single task, cancellation, and exceptions.

replaceability API , bulk operations is the next thing i will keep working on.
Thank you for your support

[charan-003]

@hkaiser @isidorostsa this version is using stdexec

I'm not sure of the failing test case, any hint on the failing cases ?

[charan-003]

@hkaiser @isidorostsa
Does the current implementation, now relying on stdexec, meet our expectations? If so, I’ll proceed with implementing bulk support. Please confirm.

[isidorostsa]

Great work! Some changes needed, but I am very happy to see you getting the hang of things!

[hkaiser]

@charan-003 please rebase your branch onto top of master to resolve the merge conflicts.

[StellarBot]

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	(=)	(=)

Info

Property	Before	After
HPX Datetime	2026-03-09T14:08:29+00:00	2026-03-24T20:16:57+00:00
HPX Commit	0eeca86	ae805e8
Envfile
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Datetime	2026-03-09T09:15:24.034803-05:00	2026-03-24T21:50:21.236223-05:00

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	++

Info

Property	Before	After
HPX Datetime	2026-03-09T14:08:29+00:00	2026-03-24T20:16:57+00:00
HPX Commit	0eeca86	ae805e8
Envfile
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Datetime	2026-03-09T09:17:15.638328-05:00	2026-03-24T21:52:02.565982-05:00

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	(=)	--
Stream Benchmark - Scale	(=)	--	--
Stream Benchmark - Triad	(=)	--	--
Stream Benchmark - Copy	(=)	+++	++

Info

Property	Before	After
HPX Datetime	2026-03-09T18:50:37+00:00	2026-03-24T20:16:57+00:00
HPX Commit	ba89f5d	ae805e8
Envfile
Clustername	rostam	rostam
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Datetime	2026-03-09T17:49:10.837937-05:00	2026-03-24T21:52:22.801134-05:00

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

[StellarBot]

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	(=)	(=)

Info

Property	Before	After
HPX Commit	0eeca86	aba2418
HPX Datetime	2026-03-09T14:08:29+00:00	2026-03-30T14:32:36+00:00
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu
Datetime	2026-03-09T09:15:24.034803-05:00	2026-03-30T10:01:23.803300-05:00
Clustername	rostam	rostam
Envfile

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	++

Info

Property	Before	After
HPX Commit	0eeca86	aba2418
HPX Datetime	2026-03-09T14:08:29+00:00	2026-03-30T14:32:36+00:00
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu
Datetime	2026-03-09T09:17:15.638328-05:00	2026-03-30T10:03:04.947346-05:00
Clustername	rostam	rostam
Envfile

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	(=)	---
Stream Benchmark - Scale	(=)	--	--
Stream Benchmark - Triad	(=)	--	--
Stream Benchmark - Copy	(=)	+++	+

Info

Property	Before	After
HPX Commit	ba89f5d	aba2418
HPX Datetime	2026-03-09T18:50:37+00:00	2026-03-30T14:32:36+00:00
Compiler	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8	/opt/apps/llvm/18.1.8/bin/clang++ 18.1.8
Hostname	medusa08.rostam.cct.lsu.edu	medusa08.rostam.cct.lsu.edu
Datetime	2026-03-09T17:49:10.837937-05:00	2026-03-30T10:03:25.284297-05:00
Clustername	rostam	rostam
Envfile

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 66 complexity · 22 duplication

Metric	Results
Complexity	66
Duplication	22

View in Codacy

_{TIP This summary will be updated as you push new changes. Give us feedback}

[hkaiser]

Just FYI: https://wg21.link/p4031