Skip to content

(POC) Add monthly benchmarking results to the changelog#8733

Open
amyblais wants to merge 1 commit intomasterfrom
amyblais-benchmarkpoc
Open

(POC) Add monthly benchmarking results to the changelog#8733
amyblais wants to merge 1 commit intomasterfrom
amyblais-benchmarkpoc

Conversation

@amyblais
Copy link
Member

@amyblais amyblais commented Feb 9, 2026

We're doing a great job benchmarking our releases and the test results are useful as they show consistent improvement. Proposal is to make the test results more visible for customer trust.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 9, 2026

Newest code from mattermost has been published to preview environment for Git SHA 47d6ac9

@amyblais amyblais changed the title Update mattermost-v11-changelog.md (POC) Add monthly benchmarking results to the changelog Feb 9, 2026
@amyblais amyblais added 1: Dev Review Requires review by a core commiter Do Not Merge Should not be merged until this label is removed 2. SME Review labels Feb 9, 2026
Copy link
Member

@agarciamontoro agarciamontoro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, @amyblais, thank you!

I know I'm not consistent in my wording in the reports, but now that we're adding it to our more visible docs, it makes sense to agree on a convention. I like the wording of the three last reports in the PR. Tweaking a couple of things to be more specific, we could use the following template:

Benchmarking test results showed no significant difference: a x% increase/decrease in the number of supported users for the new release, which lies within the [-5%, +5%] prediction interval. View the full raw data and methodology in our Performance Reports repository.

I'm open to tweaking that, but consistency is important. Also, I think that starting with "results showed no significant difference" is good to deliver the most important information first.

I'll also adapt my future reports to the template decided here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1: Dev Review Requires review by a core commiter 2. SME Review Do Not Merge Should not be merged until this label is removed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants