Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 4 additions & 52 deletions dev/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,59 +19,11 @@

# Release Process

DataFusion typically has major releases around once per month, including breaking API changes.
For contributor-facing guidance on release branches and backports, see the
[Contributor Guide Release Management page](../../docs/source/contributor-guide/release_management.md).

Patch releases are made on an adhoc basis, but we try and avoid them given the frequent major releases.

## Release Process Overview

New development happens on the `main` branch.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved this comment into a new page on the doc site for more visibility

Releases are made from branches, e.g. `branch-50` for the `50.x.y` release series.

To prepare for a new release series, we:

- Create a new branch from `main`, such as `branch-50` in the Apache repository (not in a fork)
- Continue merging new features changes to `main` branch
- Prepare the release branch for release:
- Update version numbers in `Cargo.toml` files and create `CHANGELOG.md`
- Add additional changes to the release branch as needed
- When the code is ready, create GitHub tags release candidate (rc) artifacts from the release branch.
- After the release is approved, publish to [crates.io], the ASF distribution servers, and GitHub tags.

To add changes to the release branch, depending on the change we either:

- Fix the issue on `main` and then backport the change to the release branch (e.g. [#18129])
- Fix the issue on the release branch and then forward-port the change back to `main` (e.g.[#18057])

[crates.io]: https://crates.io/crates/datafusion
[#18129]: https://github.com/apache/datafusion/pull/18129
[#18057]: https://github.com/apache/datafusion/pull/18057

## Backporting (add changes) to `branch-*` branch

If you would like to propose your change for inclusion in a patch release, the
change must be applied to the relevant release branch. To do so please follow
these steps:

1. Find (or create) the issue for the incremental release ([example release issue]) and discuss the proposed change there with the maintainers.
2. Follow normal workflow to create PR to `main` branch and wait for its approval and merge.
3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-50`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].

For example, to backport commit `12345` from `main` to `branch-50`:

```shell
git checkout branch-50
git checkout -b backport_to_50
git cherry-pick 12345 # your git commit hash
git push -u <your fork>
# make a PR as normal targeting branch-50, prefixed with [branch-50]
```

It is also acceptable to fix the issue directly on the release branch first
and then cherry-pick the change back to `main` branch in a new PR.

[example release issue]: https://github.com/apache/datafusion/issues/18072
[example backport pr]: https://github.com/apache/datafusion/pull/18131
The rest of this guide contains the commands for maintainers to create release
candidates and run the release process.

## Release Prerequisites

Expand Down
5 changes: 5 additions & 0 deletions docs/source/contributor-guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,11 @@ If you are concerned that a larger design will be lost in a string of small PRs,

Note all commits in a PR are squashed when merged to the `main` branch so there is one commit per PR after merge.

## Release Management and Backports

Contributor-facing guidance for release branches, patch releases, and backports
is documented in the [Release Management](release_management.md) guide.

## Conventional Commits & Labeling PRs

We generate change logs for each release using an automated process that will categorize PRs based on the title
Expand Down
123 changes: 123 additions & 0 deletions docs/source/contributor-guide/release_management.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Release Management

This page describes DataFusion release branches and backports. For the
maintainer release guide, including release candidate artifacts, voting, and
publication, see [the release process README in `dev/release`].

## Overview

DataFusion typically has a major release about once per month, including
breaking API changes. Patch releases are made on an ad hoc basis, but we try to
avoid them because major releases are frequent.

New development happens on the [`main` branch]. Releases are made from release
branches named `branch-NN`, such as [`branch-50`] for the `50.x.y` release
series.

In general:

- New features land on [`main`]
- Patch releases are cut from the corresponding `branch-NN`
- Only targeted, low-risk fixes should be added to a release branch

Changes reach a release branch in one of two ways:

- (Most common) Fix the issue on `main` and then backport the merged change to the release branch
- Fix the issue on the release branch and then forward-port the change to `main`

Releases are coordinated in a GitHub issue, such as the
[release issue for 50.3.0]. If you think a fix should be included in a patch
release, discuss it on the relevant tracking issue first. You can also open the
backport PR first and then link it from the tracking issue.

To prepare for a new release series, maintainers:

- Create a new branch from `main`, such as `branch-50`, in the Apache repository
- Continue merging new features to `main`
- Prepare the release branch for release by updating versions, changelog content,
and any additional release-specific fixes via the
[Backport Workflow](#backport-workflow)
- Create release candidate artifacts from the release branch
- After approval, publish to crates.io, ASF distribution servers, and Git tags

## Backport Workflow

The usual workflow is:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from my experience if backport(s) happened, should we also create a new tag, new RC and new voting process?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and also update the change lists


1. Fix on `main` first, and merge the fix via a normal PR workflow.
2. Cherry-pick the merged commit onto the release branch.
3. Open a backport PR targeting the release branch (examples below).

- [Example backport PR]
- [Additional backport PR example]

### Inputs

To backport a change, gather the following information:

- Target branch, such as `apache/branch-52`
- The release tracking issue URL, such as https://github.com/apache/datafusion/issues/19692
- The original PR URL, such as https://github.com/apache/datafusion/pull/20192
- Optional explicit commit SHA to backport

### Apply the Backport

Start from the target release branch, create a dedicated backport branch, and
use `git cherry-pick`. For example, to backport PR #1234 to `branch-52` when
the commit SHA is `abc123`, run:

```bash
git checkout apache/branch-52
git checkout -b alamb/backport_1234
git cherry-pick abc123
```

### Test

Run tests as described in the [testing documentation].

### Open the PR

Create a PR against the release branch, not `main`, and prefix it with
`[branch-NN]` to show which release branch the backport targets. For example:

- `[branch-52] fix: validate inter-file ordering in eq_properties() (#20329)`

Use a PR description that links the tracking issue, original PR, and target
branch, for example:

```markdown
- Part of <tracking-issue-url>
- Closes <backport-issue-url> on <branch-name>

This PR:

- Backports <original-pr-url> from @<author> to the <branch-name> line
```

[`main` branch]: https://github.com/apache/datafusion/tree/main
[`branch-50`]: https://github.com/apache/datafusion/tree/branch-50
[the release process readme in `dev/release`]: https://github.com/apache/datafusion/blob/main/dev/release/README.md
[release issue for 50.3.0]: https://github.com/apache/datafusion/issues/18072
[example backport pr]: https://github.com/apache/datafusion/pull/18131
[additional backport pr example]: https://github.com/apache/datafusion/pull/20792
[testing documentation]: testing.md
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ To get started, see
contributor-guide/testing
contributor-guide/api-health
contributor-guide/howtos
contributor-guide/release_management
contributor-guide/roadmap
contributor-guide/governance
contributor-guide/inviting
Expand Down
Loading