Backup to S3 Object ACL Issues

SingleStore Tools MemSQL Ops
#1

Hey All,

I was wondering if anyone has ever done backups to an s3 bucket that exists in a different AWS account? I’m running into issues with S3 Object ACLs. The files that are being uploaded from AWS Account A to the S3 Bucket in AWS Account B are restricted to the AWS account that uploaded the objects (in this case AWS Account A). The Bucket owner (AWS Account B) does not have permissions to access the files. I spoke with AWS Support and the --acl bucket-owner-full-control parameter needs to be included to give the bucket owner in AWS account B full access to those objects. I there any way to configure this when using the BACKUP command in memsql?

Thanks,
Aaron

#2

Also, can the BACKUP run without including the CREDENTIALS parameter? Or is it possible to just have a bucket policy that gives access to the bucket?

#3

Hi Aaron,

For your first question, currently we do not have the --acl bucket-owner-full-control parameter set when doing the backup. A workaround for this would be to have your policy set, so that any uploads to a bucket give full control to the bucket owner. Amazon has a good example of how to do this here: https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-8

For the second, you can pass empty credentials, however syntactically you still need to provide a credentials parameter, you can do this by just passing ‘{}’ as the credential string.

Modifying the example found in the docs, it would look like this:

BACKUP DATABASE memsql_demo TO S3 "backup_bucket/backups/6_1_2018" CONFIG '{"region":"us-east-1"}' CREDENTIALS '{}'

I hope that helps!
Nate