Skip to content

Feature: Reorganize CI for faster feedback and domain-split test jobs #203

@vkozio

Description

@vkozio

API

Other

Description

Problem

CI runs feel long and monolithic. When something breaks, we often wait for a full minimal test or minimal linux extension test job to finish. Both jobs do a full build and a large set of tests in one go, so it’s hard to get quick, targeted feedback (e.g. “only storage tests failed”).

Current shape (PRs)

  • Format/sanity jobs (clang-format, python-format, rust-format, sanity-checks) are already parallel and quick.
  • Two heavy jobs run in parallel:
    • minimal test: sanity → full build (relwithdebinfo) → generate datasets → make shell-test test
    • minimal linux extension test: full build + extension-test-build + extension tests
      Each does its own build; neither is split by domain, so a failure in one area still means waiting for the whole job.

Proposal (for maintainers to consider)

  1. Domain-split test jobs
    Keep one build (or one build job that publishes an artifact). Add several test jobs that consume that build and run different subsets, e.g.:

    • Core / unit tests (e2e or gtest with filters for storage, transaction, etc.)
    • Shell / main e2e tests
    • Extension tests
      So a failure in one area fails only that job and shows up in ~5–15 minutes instead of the full pipeline.
  2. Single build, multiple test jobs
    One job: build (relwithdebinfo) + generate datasets → upload artifact. Other jobs: download artifact and run only their test subset. Build cost is paid once; test jobs can run in parallel.

  3. Tiered CI (optional)

    • Tier 1 (required for merge): format, sanity, build, and a small smoke set. Fast feedback for “can we merge?”
    • Tier 2: full e2e, extension, and (if re-enabled) multi-platform jobs. Could run on every PR but not block merge, or only on push to master / workflow_dispatch, so heavy runs don’t block every contributor.
  4. Path-based or optional heavy jobs
    Run extension tests only when extension/ (or related) code changes; run full self-hosted / Windows / macOS matrix only when needed (e.g. label or manual trigger). Reduces wait time when working on unrelated changes.

Why this helps

  • Faster feedback: Failures are scoped to a domain (e.g. “extension tests” vs “core tests”), so we see what broke without waiting for the entire pipeline.
  • Clearer signals: Required checks stay small and fast; heavier or flaky jobs can be non-blocking or optional.
  • Better use of parallelism: More, smaller jobs instead of fewer, huge ones often improves wall-clock time and resource usage.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions