How to Set Up FTP Access to an Amazon S3 Bucket Using EC2

Introduction

Amazon S3 (Simple Storage Service) is a highly scalable and durable object storage service that provides easy-to-use management features. However, Amazon S3 does not natively support FTP (File Transfer Protocol) access. In this article, we will walk you through setting up FTP access to an S3 bucket using an EC2 (Elastic Compute Cloud) instance and the vsftpd server.

Prerequisites

Before you begin, you should have the following:

Step 1: Create an S3 Bucket

  1. Log in to the AWS Management Console.
  2. Navigate to the S3 Console.
  3. Click Create bucket.
  4. Provide a unique name for your bucket and select a region.
  5. Configure additional settings as needed and click Create bucket.

For more information on creating an S3 bucket, refer to the official documentation.

Step 2: Set Up an EC2 Instance

  1. Go to the EC2 Console.
  2. Click Launch Instance.
  3. Choose an Amazon Linux 2 AMI (Amazon Machine Image).
  4. Select an instance type (e.g., t2.micro) and click Next: Configure Instance Details.
  5. Configure instance details as needed and click Next: Add Storage.
  6. Add storage as required and click Next: Add Tags.
  7. Add any tags you need and click Next: Configure Security Group.
  8. Create a new security group and add rules to allow SSH (port 22) and FTP (port 21) traffic. Click Review and Launch.
  9. Review your instance settings and click Launch. 
  10. Choose an existing key pair or create a new one, and then click Launch Instances.

For more information on launching an EC2 instance, refer to the official documentation.

Step 3: Install and Configure vsftpd

1. SSH into your EC2 instance:

ssh -i <your-key-pair.pem> ec2-user@<your-instance-public-ip>

2. Update the package repository and install vsftpd:

sudo yum update -y
sudo yum install vsftpd -y

3. Open the vsftpd configuration file with a text editor (e.g., nano or vi):

sudo nano /etc/vsftpd/vsftpd.conf

4. Update the following settings in the configuration file:

anonymous_enable=NO
local_enable=YES
chroot_local_user=YES
allow_writeable_chroot=YES
pasv_enable=YES
pasv_min_port=1024
pasv_max_port=1048

5. Save and close the configuration file.

6. Enable and start the vsftpd service:

sudo systemctl enable vsftpd
sudo systemctl start vsftpd

Step 4: Configure AWS CLI

1. Install the AWS CLI on your EC2 instance:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

2. Configure the AWS CLI with your access key and secret key:

aws configure

You will be prompted to provide your AWS access key ID, secret access key, default region name, and default output format.

Step 5: Set Up FTP Scripts

1. Create a new directory for the FTP scripts:

mkdir ~/ftp_scripts

2. Create a script to upload files from the EC2 instance to the S3 bucket:

nano ~/ftp_scripts/s3_upload.sh

Add the following content to the script:

#!/bin/bash
aws s3 cp /home/ec2-user/ftp/files/ s3://<your-bucket-name>/ --recursive

2. Replace <your-bucket-name> with the name of your S3 bucket. Save and close the script file.

3. Make the script executable: chmod +x ~/ftp_scripts/s3_upload.sh

4. Set up a cron job to run the script every 5 minutes:

crontab -e

Add the following line to the crontab file:

*/5 * * * * /home/ec2-user/ftp_scripts/s3_upload.sh

5. Save and close the crontab file.

Step 6: Test FTP Access

  1. On your local machine, install an FTP client (e.g., FileZilla).
  2. Connect to your EC2 instance using the FTP client:
    • Host: <your-instance-public-ip>
    • Username: ec2-user
    • Password: (Leave this field blank)
    • Port: 21
  3. Upload files to the /home/ec2-user/ftp/files/ directory on your EC2 instance.
  4. Wait for the cron job to run and check your S3 bucket to verify that the files have been uploaded successfully.
  5. You can now essentially access your Amazon S3 using any FTP client.

Troubleshooting Tips

If you encounter issues while setting up FTP access, consider the following troubleshooting tips:

  1. Check the vsftpd logs for any error messages: sudo tail /var/log/vsftpd.log
  2. Ensure that the FTP ports (21, and 1024-1048) are open in your EC2 instance’s security group.
  3. Check the AWS CLI configuration for any errors: aws configure list
  4. If using an instance role (recommended) instead of access keys, verify that the EC2 instance has the necessary IAM permissions to access the S3 bucket.

Conclusion

In this article, we walked you through the process of setting up FTP access to an Amazon S3 bucket using an EC2 instance and the vsftpd server. By following these steps, you should now have a fully functional FTP server that uploads files to your S3 bucket.

While this solution is suitable for PoC and simple use cases, consider using more advanced solutions such as AWS Transfer Family for production environments or when you require additional features such as SFTP, FTPS, or more granular control over user access.

Related Reading

Securing FTP Transfers to Amazon S3