Setup Backup To S3 (B2S3) Through a Proxy In Your AWS VPC

Introduction

When utilizing Zadara's Backup to S3 feature from a VPSA attached to your Amazon VPC via Direct Connect, it is often desirable to use a proxy.  This achieves two goals:

  • The VPSA does not need to send backup traffic (encrypted) over the public Internet.  This improves performance.
  • Since the traffic to S3 will come from your VPC instead of from the Internet, there will be reduced costs for reads (backup restorations) compared to the normal Direct Connect costs as long as the S3 bucket is created in the same region as your VPC.

Visually, this looks like:

B2S3_Proxy.png

For the purposes of this document, we will use an authenticationless Squid 3 (controlling access via source IP with a Security Group) on Ubuntu Linux 14.04 HVM, but any https proxy server that supports the CONNECT method should work.

Setup EC2 Instance

First, launch a new Ubuntu 14.04 HVM EC2 instance in the VPC that is connected to the VPSA (you may want to enable a public IP so that you may easily access the instance later via SSH).  For instance type, the higher network performance value, the more throughput to S3 you may achieve.  The instance will not need a lot of local storage, so the default amount (8GB as of this writing) should be ok.

When setting up the Security Group, you will want to allow remote management access (SSH) as well as proxy server access only from your VPSA IP(s) to enhance security.  Using Squid's default port of 3128, an example would be:

security_group.png

Where "10.10.1.67" is the IP address of your VPSA - the "/32" specifies to only allow from this one address.  If you need to find the VPSA's address, you can do so via your management portal:

ecommerce_ip.png

Setup Squid

Once the EC2 instance is setup and an SSH session has been established, you can now install Squid 3:

sudo apt-get -y install squid3

After Squid 3 is installed, you need to add some configurations to /etc/squid3/squid.conf.  Please add them around the same section as the example acls (these need to come before the later entered blanket deny acl). Substituting the IP in the example with your VPSA's IP:

acl vpsa src 10.10.1.67/32

acl s3 dstdom_regex .*s3\.amazonaws\.com
acl s3 dstdom_regex .*s3\.eu-central-1\.amazonaws\.com
acl s3 dstdom_regex .*s3\.sa-east-1\.amazonaws\.com
acl s3 dstdom_regex .*s3\.ap-northeast-1\.amazonaws\.com
acl s3 dstdom_regex .*s3\.eu-west-1\.amazonaws\.com
acl s3 dstdom_regex .*s3\.us-west-1\.amazonaws\.com
acl s3 dstdom_regex .*s3\.us-west-2\.amazonaws\.com
acl s3 dstdom_regex .*s3\.ap-southeast-2\.amazonaws\.com
acl s3 dstdom_regex .*s3\.ap-southeast-1\.amazonaws\.com

acl s3 dstdom_regex .*s3-eu-central-1\.amazonaws\.com
acl s3 dstdom_regex .*s3-sa-east-1\.amazonaws\.com
acl s3 dstdom_regex .*s3-ap-northeast-1\.amazonaws\.com
acl s3 dstdom_regex .*s3-eu-west-1\.amazonaws\.com
acl s3 dstdom_regex .*s3-us-west-1\.amazonaws\.com
acl s3 dstdom_regex .*s3-us-west-2\.amazonaws\.com
acl s3 dstdom_regex .*s3-ap-southeast-2\.amazonaws\.com
acl s3 dstdom_regex .*s3-ap-southeast-1\.amazonaws\.com

http_access allow vpsa s3

After saving this configuration, restart the squid3 service:

sudo service squid3 restart

Connect S3 Endpoint In VPSA GUI

Finally, now that the proxy is setup, the S3 endpoint can be discovered and S3 backups can be setup and commence running.

To discover a VPSA endpoint, log into the VPSA GUI and click "Remote Object Storage", then "Connect":

 initiate_connect_to_s3.png

Now fill in all the appropriate information for your S3 configuration:

connect_to_s3_settings.png

If you need help finding the private IP of the EC2 instance, highlight the instance in the EC2 console, then view the properties underneath:

private_ip.png

S3 Permissions

The following permissions are required for the provided IAM user for the bucket in question:

  • List*
  • GetObject
  • PutObject
  • DeleteObject

Troubleshooting

If you have trouble connecting to the S3 bucket, please contact support@zadarastorage.com for further assistance.

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.