Using HeadBucket for backups is dangerous in multi-tenant S3 buckets

[DimitriosLisenko]

Proposal

This project uses HeadBucket to determine the signature version to use, as well as whether the bucket exists before creating backups. The problem with this is that in IAM, there is literally no way to let a user call HeadBucket while also preventing them from listing objects inside that bucket. This is because HeadBucket requires the s3:ListBucket permission.

There are zero air-tight IAM workarounds that would prevent this permission from also allowing users to list objects inside the bucket. Here are some examples:

1. Grant s3:ListBucket conditionally on the "" prefix - does not allow HeadBucket because HeadBucket does not pass a prefix parameter, and so it is denied.
2. Grant s3:ListBucket conditionally on a specific directory - same as above.
3. Grant s3:ListBucket conditionally on "Null": {"s3:prefix": "true"} - this actually still allows listing all objects in the bucket, just not specifying a prefix.
4. Grant s3:ListBucket conditionally on everything except */* - can still specify a prefix without a slash and list objects.

There has been extensive discussion around this issue as it relates to the CNPG plugin:
EnterpriseDB/barman#929
EnterpriseDB/barman#1101
hashicorp/terraform-provider-aws#17195

The initial suggestion was to use ListObjectsV2 with a prefix rather than HeadBucket - this works in terms of security, but it is more expensive in terms of AWS API calls, and so was rejected.

The final solution as it relates to CNPG was to completely remove cloud connectivity + bucket existence checks before running backups. I propose the same here, which would require to remove the signature checks and require that it is provider; or instead to use HeadObject, which can be scoped to a directory.

Use-Case

The use case is for multi-tenant S3 buckets. Using HeadBucket in this scenario is a non-starter because it would allow users to see backups of other users.

Anything else?

This is a requirement to resolve percona/percona-xtradb-cluster-operator#2251, because that operator calls HeadBucket in itself, as well as by virtue of calling xbcloud.

[it-percona-cla]

All committers have signed the CLA.

[DimitriosLisenko]

This PR is required for HeadBucket to be removed from the backup flow, and also for the operator to stop using HeadBucket. The operator work is currently at percona/percona-xtradb-cluster-operator#2326 - are the same people able to review this item as well?

[egegunes]

@satya-bodapati could you please take a look when you have the chance?