API
Other
Description
Problem
CI runs feel long and monolithic. When something breaks, we often wait for a full minimal test or minimal linux extension test job to finish. Both jobs do a full build and a large set of tests in one go, so it’s hard to get quick, targeted feedback (e.g. “only storage tests failed”).
Current shape (PRs)
- Format/sanity jobs (clang-format, python-format, rust-format, sanity-checks) are already parallel and quick.
- Two heavy jobs run in parallel:
- minimal test: sanity → full build (relwithdebinfo) → generate datasets →
make shell-test test
- minimal linux extension test: full build + extension-test-build + extension tests
Each does its own build; neither is split by domain, so a failure in one area still means waiting for the whole job.
Proposal (for maintainers to consider)
-
Domain-split test jobs
Keep one build (or one build job that publishes an artifact). Add several test jobs that consume that build and run different subsets, e.g.:
- Core / unit tests (e2e or gtest with filters for storage, transaction, etc.)
- Shell / main e2e tests
- Extension tests
So a failure in one area fails only that job and shows up in ~5–15 minutes instead of the full pipeline.
-
Single build, multiple test jobs
One job: build (relwithdebinfo) + generate datasets → upload artifact. Other jobs: download artifact and run only their test subset. Build cost is paid once; test jobs can run in parallel.
-
Tiered CI (optional)
- Tier 1 (required for merge): format, sanity, build, and a small smoke set. Fast feedback for “can we merge?”
- Tier 2: full e2e, extension, and (if re-enabled) multi-platform jobs. Could run on every PR but not block merge, or only on push to
master / workflow_dispatch, so heavy runs don’t block every contributor.
-
Path-based or optional heavy jobs
Run extension tests only when extension/ (or related) code changes; run full self-hosted / Windows / macOS matrix only when needed (e.g. label or manual trigger). Reduces wait time when working on unrelated changes.
Why this helps
- Faster feedback: Failures are scoped to a domain (e.g. “extension tests” vs “core tests”), so we see what broke without waiting for the entire pipeline.
- Clearer signals: Required checks stay small and fast; heavier or flaky jobs can be non-blocking or optional.
- Better use of parallelism: More, smaller jobs instead of fewer, huge ones often improves wall-clock time and resource usage.
API
Other
Description
Problem
CI runs feel long and monolithic. When something breaks, we often wait for a full
minimal testorminimal linux extension testjob to finish. Both jobs do a full build and a large set of tests in one go, so it’s hard to get quick, targeted feedback (e.g. “only storage tests failed”).Current shape (PRs)
make shell-test testEach does its own build; neither is split by domain, so a failure in one area still means waiting for the whole job.
Proposal (for maintainers to consider)
Domain-split test jobs
Keep one build (or one build job that publishes an artifact). Add several test jobs that consume that build and run different subsets, e.g.:
So a failure in one area fails only that job and shows up in ~5–15 minutes instead of the full pipeline.
Single build, multiple test jobs
One job: build (relwithdebinfo) + generate datasets → upload artifact. Other jobs: download artifact and run only their test subset. Build cost is paid once; test jobs can run in parallel.
Tiered CI (optional)
master/workflow_dispatch, so heavy runs don’t block every contributor.Path-based or optional heavy jobs
Run extension tests only when
extension/(or related) code changes; run full self-hosted / Windows / macOS matrix only when needed (e.g. label or manual trigger). Reduces wait time when working on unrelated changes.Why this helps