Amazon S3 (Simple Storage Service) is a highly scalable and durable object storage service that provides easy-to-use management features. However, Amazon S3 does not natively support FTP (File Transfer Protocol) access. In this article, we will walk you through setting up FTP access to an S3 bucket using an EC2 (Elastic Compute Cloud) instance and the vsftpd server.
Before you begin, you should have the following:
- An active Amazon Web Services (AWS) account
- AWS CLI (Command Line Interface) installed and configured on your local machine
- Basic knowledge of Amazon S3 and Amazon EC2
Step 1: Create an S3 Bucket
- Log in to the AWS Management Console.
- Navigate to the S3 Console.
- Click Create bucket.
- Provide a unique name for your bucket and select a region.
- Configure additional settings as needed and click Create bucket.
For more information on creating an S3 bucket, refer to the official documentation.
Step 2: Set Up an EC2 Instance
- Go to the EC2 Console.
- Click Launch Instance.
- Choose an Amazon Linux 2 AMI (Amazon Machine Image).
- Select an instance type (e.g., t2.micro) and click Next: Configure Instance Details.
- Configure instance details as needed and click Next: Add Storage.
- Add storage as required and click Next: Add Tags.
- Add any tags you need and click Next: Configure Security Group.
- Create a new security group and add rules to allow SSH (port 22) and FTP (port 21) traffic. Click Review and Launch.
- Review your instance settings and click Launch.
- Choose an existing key pair or create a new one, and then click Launch Instances.
For more information on launching an EC2 instance, refer to the official documentation.
Step 3: Install and Configure vsftpd
1. SSH into your EC2 instance:
ssh -i <your-key-pair.pem> ec2-user@<your-instance-public-ip>
2. Update the package repository and install vsftpd:
sudo yum update -y sudo yum install vsftpd -y
3. Open the vsftpd configuration file with a text editor (e.g., nano or vi):
sudo nano /etc/vsftpd/vsftpd.conf
4. Update the following settings in the configuration file:
anonymous_enable=NO local_enable=YES chroot_local_user=YES allow_writeable_chroot=YES pasv_enable=YES pasv_min_port=1024 pasv_max_port=1048
5. Save and close the configuration file.
6. Enable and start the vsftpd service:
sudo systemctl enable vsftpd sudo systemctl start vsftpd
Step 4: Configure AWS CLI
1. Install the AWS CLI on your EC2 instance:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" unzip awscliv2.zip sudo ./aws/install
2. Configure the AWS CLI with your access key and secret key:
You will be prompted to provide your AWS access key ID, secret access key, default region name, and default output format.
Step 5: Set Up FTP Scripts
1. Create a new directory for the FTP scripts:
2. Create a script to upload files from the EC2 instance to the S3 bucket:
Add the following content to the script:
#!/bin/bash aws s3 cp /home/ec2-user/ftp/files/ s3://<your-bucket-name>/ --recursive
<your-bucket-name> with the name of your S3 bucket. Save and close the script file.
3. Make the script executable:
chmod +x ~/ftp_scripts/s3_upload.sh
4. Set up a cron job to run the script every 5 minutes:
Add the following line to the crontab file:
*/5 * * * * /home/ec2-user/ftp_scripts/s3_upload.sh
5. Save and close the crontab file.
Step 6: Test FTP Access
- On your local machine, install an FTP client (e.g., FileZilla).
- Connect to your EC2 instance using the FTP client:
- Password: (Leave this field blank)
- Upload files to the
/home/ec2-user/ftp/files/directory on your EC2 instance.
- Wait for the cron job to run and check your S3 bucket to verify that the files have been uploaded successfully.
- You can now essentially access your Amazon S3 using any FTP client.
If you encounter issues while setting up FTP access, consider the following troubleshooting tips:
- Check the vsftpd logs for any error messages:
sudo tail /var/log/vsftpd.log
- Ensure that the FTP ports (21, and 1024-1048) are open in your EC2 instance’s security group.
- Check the AWS CLI configuration for any errors:
aws configure list
- If using an instance role (recommended) instead of access keys, verify that the EC2 instance has the necessary IAM permissions to access the S3 bucket.
In this article, we walked you through the process of setting up FTP access to an Amazon S3 bucket using an EC2 instance and the vsftpd server. By following these steps, you should now have a fully functional FTP server that uploads files to your S3 bucket.
While this solution is suitable for PoC and simple use cases, consider using more advanced solutions such as AWS Transfer Family for production environments or when you require additional features such as SFTP, FTPS, or more granular control over user access.