Add tool to fix missing CRR replication permissions by nicolas2bert · Pull Request #377 · scality/s3utils

nicolas2bert · 2026-02-27T20:42:25Z

Add fix-missing-replication-permissions.js that reads the output of check-replication-permissions.js and automatically creates per-bucket IAM policies with s3:ReplicateObject for roles that are missing it.

The script is

idempotent
supports --dry-run,
cleans up temporary credentials after use.

Switch from $'...' quoting with \n to <br> for line breaks in comment bodies, except inside fenced code blocks.

…eanup Include ownerDisplayName in check-replication-permissions.js results so the fix script can map accounts without extra API calls. Improve deleteTestAccount to clean up all buckets in the account (not just the initial one) and delete detached policies.

New script reads the output of check-replication-permissions.js and creates per-bucket IAM policies with s3:ReplicateObject, then attaches them to the corresponding roles. Supports --dry-run and is idempotent. Move @aws-sdk/client-iam to production dependencies (required at runtime). Add functional tests covering dry-run, idempotency, multi-bucket, multi-role, multi-account, key cleanup, and input validation.

bert-e · 2026-02-27T20:42:28Z

Hello nicolas2bert,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-02-27T20:42:33Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

replicationAudit/fix-missing-replication-permissions.js

claude · 2026-02-27T20:45:53Z

LGTM

Review by Claude Code

codecov · 2026-02-27T20:52:10Z

Codecov Report

❌ Patch coverage is 0% with 136 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.67%. Comparing base (378f68f) to head (92d0e98).
⚠️ Report is 7 commits behind head on development/1.17.

Files with missing lines	Patch %	Lines
...cationAudit/fix-missing-replication-permissions.js	0.00%	132 Missing and 1 partial ⚠️
replicationAudit/check-replication-permissions.js	0.00%	3 Missing ⚠️

Additional details and impacted files

@@                 Coverage Diff                  @@
##           development/1.17     #377      +/-   ##
====================================================
- Coverage             43.62%   42.67%   -0.96%     
====================================================
  Files                    84       85       +1     
  Lines                  5973     6106     +133     
  Branches               1255     1270      +15     
====================================================
  Hits                   2606     2606              
- Misses                 3321     3453     +132     
- Partials                 46       47       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

replicationAudit/fix-missing-replication-permissions.js

replicationAudit/README.md

scality-gdoumergue · 2026-03-02T09:43:44Z

replicationAudit/README.md

+# Step 7 (optional): Fix missing permissions
+# Run from your local machine (requires vaultclient and @aws-sdk/client-iam)
+node replicationAudit/fix-missing-replication-permissions.js \
+    /root/replicationAudit_missing.json <supervisor-ip> admin1.json


Move this step before the cleanup step, and run the fix script with ansible:
(previous code was running the nodeJS script from the supervisor, which would fail)

Suggested change

# Step 7 (optional): Fix missing permissions

# Run from your local machine (requires vaultclient and @aws-sdk/client-iam)

node replicationAudit/fix-missing-replication-permissions.js \

/root/replicationAudit_missing.json <supervisor-ip> admin1.json

# Step 6 (optional): Fix missing permissions

ansible -i env/$ENV_DIR/inventory runners_s3[0] -m copy \

-a "src=env/$ENV_DIR/vault/admin-clientprofile/admin1.json dest={{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs"

ansible -i env/$ENV_DIR/inventory runners_s3[0] -m shell \

-a "ctrctl exec scality-vault{{ container_name_suffix | default("")}} \

node /logs/fix-missing-replication-permissions.js /logs/missing.json 127.0.0.1 /logs/admin1.json"

scality-gdoumergue · 2026-03-02T09:44:47Z

replicationAudit/README.md

+# Step 6: Clean up remote files
 ansible -i env/$ENV_DIR/inventory runners_s3[0] -m shell \
    -a 'rm -f {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/missing.json \
       {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/check-replication-permissions.js \
       {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/buckets-with-replication.json \
       /root/list-buckets-with-replication.sh'


Suggested change

# Step 6: Clean up remote files

ansible -i env/$ENV_DIR/inventory runners_s3[0] -m shell \

-a 'rm -f {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/missing.json \

{{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/check-replication-permissions.js \

{{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/buckets-with-replication.json \

/root/list-buckets-with-replication.sh'

# Step 7: Clean up remote files

ansible -i env/$ENV_DIR/inventory runners_s3[0] -m shell \

-a 'rm -f {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/missing.json \

{{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/check-replication-permissions.js \

{{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/buckets-with-replication.json \

{{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/fix-missing-replication-permissions.js \

{{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/admin1.json \

/root/list-buckets-with-replication.sh'

replicationAudit/README.md

Switch from @aws-sdk/client-iam v3 to aws-sdk v2 so the fix script can run inside the vault container of older S3C versions (pre-9.5.2) where only v2 is available. Update README workflow to copy and execute the fix script inside the vault container instead of running it externally.

replicationAudit/fix-missing-replication-permissions.js

+        sslEnabled: config.useHttps,
+        httpOptions: {
+            agent: config.useHttps
+                ? new https.Agent({ keepAlive: true, rejectUnauthorized: false })


In general, the fix is to stop disabling TLS certificate validation and instead rely on the default behavior (or explicitly set rejectUnauthorized: true). If the environment uses self‑signed or private CA certificates, those should be trusted via the OS/Node trust store or by providing a CA bundle to the HTTPS agent, not by turning validation off.

Concretely for this file, in createIAMClient, we should change the HTTPS agent construction so it no longer sets rejectUnauthorized: false. The safest minimal change that preserves existing behavior (keep‑alive, optional HTTPS) is:

For HTTPS: use new https.Agent({ keepAlive: true }), allowing Node’s default certificate validation.

Optionally, if this script must support a custom CA bundle, we could add a ca option wired from configuration, but the prompt doesn’t show such config, so we’ll keep the change minimal.

Only replicationAudit/fix-missing-replication-permissions.js needs editing. No new imports or helper methods are required; we simply modify the httpOptions.agent initialization lines 130–133.

claude · 2026-03-03T14:33:45Z

LGTM

Review by Claude Code

scality-gdoumergue · 2026-03-03T15:21:31Z

replicationAudit/README.md

We need to keep /root/buckets-with-replication.json so that we can re-run checks

Suggested change

-a "cp /root/buckets-with-replication.json {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs && \

scality-gdoumergue · 2026-03-03T15:26:53Z

replicationAudit/README.md

       {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/buckets-with-replication.json \
+       {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/replication-fix-results.json \
+       {{ env_host_logs}}/scality-vault{{ container_name_suffix | default("")}}/logs/admin1.json \
       /root/list-buckets-with-replication.sh'


Suggested change

/root/list-buckets-with-replication.sh'

/root/list-buckets-with-replication.sh /root/buckets-with-replication.json'

claude · 2026-03-03T16:31:11Z

LGTM

Review by Claude Code

…ecks Use cp instead of mv so the bucket list file remains at /root/ on the runner, allowing steps 3-5 to be repeated without re-running the list script.

claude · 2026-03-03T16:42:11Z

LGTM

Review by Claude Code

Add GHSA-j965-2qgj-vjmq to the allow list since aws-sdk v2 is intentionally used as a devDependency for vault container compatibility. Also add /root/buckets-with-replication.json to step 8 cleanup.

claude · 2026-03-03T16:51:28Z

LGTM

Review by Claude Code

nicolas2bert · 2026-03-04T12:26:21Z

@bert-e approve

bert-e · 2026-03-04T12:26:32Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge improvement/S3UTILS-222/fix-crr-policy into
development/1.17
w/1/improvement/S3UTILS-222/fix-crr-policy will be merged into development/1

The following branches will NOT be impacted:

development/1.13
development/1.14
development/1.15
development/1.16
development/1.4
development/1.7

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: approve

claude · 2026-03-04T12:27:49Z

LGTM

Review by Claude Code

bert-e · 2026-03-04T12:37:05Z

Build failed

The build for commit did not succeed in branch improvement/S3UTILS-222/fix-crr-policy

The following options are set: approve

bert-e · 2026-03-09T12:20:07Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/1.17
✔️ development/1

The following branches have NOT changed:

development/1.13
development/1.14
development/1.15
development/1.16
development/1.4
development/1.7

Please check the status of the associated issue S3UTILS-222.

Goodbye nicolas2bert.

The following options are set: approve

nicolas2bert added 3 commits February 27, 2026 21:36

S3UTILS-223: fix review-pr skill quoting for inline comments

2a3a45f

Switch from $'...' quoting with \n to <br> for line breaks in comment bodies, except inside fenced code blocks.

github-advanced-security bot found potential problems Feb 27, 2026

View reviewed changes

replicationAudit/fix-missing-replication-permissions.js Fixed Show fixed Hide fixed

nicolas2bert requested a review from scality-gdoumergue February 27, 2026 20:44

scality-gdoumergue requested changes Mar 2, 2026

View reviewed changes

github-advanced-security bot found potential problems Mar 3, 2026

View reviewed changes

scality-gdoumergue requested changes Mar 3, 2026

View reviewed changes

S3UTILS-222: preserve buckets-with-replication.json for re-running ch…

1795d54

…ecks Use cp instead of mv so the bucket list file remains at /root/ on the runner, allowing steps 3-5 to be repeated without re-running the list script.

nicolas2bert force-pushed the improvement/S3UTILS-222/fix-crr-policy branch from f7bd3a9 to 1795d54 Compare March 3, 2026 16:38

S3UTILS-222: allow low-severity aws-sdk v2 advisory in dependency review

e0af0fa

Add GHSA-j965-2qgj-vjmq to the allow list since aws-sdk v2 is intentionally used as a devDependency for vault container compatibility. Also add /root/buckets-with-replication.json to step 8 cleanup.

scality-gdoumergue approved these changes Mar 4, 2026

View reviewed changes

leif-scality approved these changes Mar 4, 2026

View reviewed changes

S3UTILS-222 bump package.json

92d0e98

BourgoisMickael approved these changes Mar 4, 2026

View reviewed changes

bert-e merged commit 92d0e98 into development/1.17 Mar 9, 2026
13 of 16 checks passed

bert-e deleted the improvement/S3UTILS-222/fix-crr-policy branch March 9, 2026 12:20


	-a "cp /root/buckets-with-replication.json {{ env_host_logs}}/scality-vault{{ container_name_suffix \| default("")}}/logs && \

	/root/list-buckets-with-replication.sh'
	/root/list-buckets-with-replication.sh /root/buckets-with-replication.json'

Conversation

nicolas2bert commented Feb 27, 2026

Uh oh!

bert-e commented Feb 27, 2026

Hello nicolas2bert,

Uh oh!

bert-e commented Feb 27, 2026

Request integration branches

Uh oh!

Uh oh!

claude bot commented Feb 27, 2026

Uh oh!

codecov bot commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

scality-gdoumergue Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

scality-gdoumergue Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Check failure

Copilot Autofix

claude bot commented Mar 3, 2026

Uh oh!

scality-gdoumergue Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

scality-gdoumergue Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

claude bot commented Mar 3, 2026

Uh oh!

claude bot commented Mar 3, 2026

Uh oh!

claude bot commented Mar 3, 2026

Uh oh!

nicolas2bert commented Mar 4, 2026

Uh oh!

bert-e commented Mar 4, 2026

Integration data created

Uh oh!

claude bot commented Mar 4, 2026

Uh oh!

bert-e commented Mar 4, 2026

Build failed

Uh oh!

bert-e commented Mar 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented Feb 27, 2026 •

edited

Loading