Developers and IT teams have access to a popular object storage service called AWS S3, which offers safe, scalable, and highly accessible data storage. Nevertheless, utilizing the AWS Command Line Interface (CLI) to access the service can occasionally result in issues. Troubleshooting these problems can be irritating and time-consuming, especially for people who are unfamiliar with AWS and its command-line interface. A list of typical errors is provided below:
- AccessDenied
- NoSuchBucket
- NoSuchKey/ Key does not exist
- InvalidArgument
- BucketAlreadyExists
- NoSuchBucketPolicy
- SlowDown
- AuthFailure
In this blog, we will discuss these common errors that you may encounter while using AWS CLI to access the AWS S3 service and provide step-by-step instructions on how to diagnose and fix them. We’ll cover everything from authentication issues to bucket policy issues and provide tips on how to prevent these errors from happening in the future. Whether you’re an AWS pro or just starting out, this blog is for you. Let’s dive in and learn how to troubleshoot S3 errors on AWS CLI!
1. AccessDenied
Error Message: upload failed: ./hello.txt to s3://my-k21-bucket/hello.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Incorrect method:
aws s3 cp hello.txt s3://my-k21-bucket
Explanation:
The above AWS CLI command is trying to upload a file “hello.txt ” to the S3 bucket “my-k21-bucket”. However, if the AWS CLI user does not have permission to access the S3 bucket, this command will result in an “AccessDenied” error.
Solution:
Verify your S3 bucket policies and that the IAM user or role you are using to upload the object has the required permissions before uploading the object. The IAM user or role must have the appropriate rights to upload objects to the bucket, so check the bucket policy and the access control list (ACL) to make sure.
Additionally, the user or IAM role that is executing the AWS CLI command needs to have the PutObject permission. By including the following policy statement to the user or IAM role, you can do this:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowBucketPutObject", "Effect": "Allow", "Action": "s3:PutObject", "Resource": "arn:aws:s3:::my-k21-bucket/*" } ] }
2. NoSuchBucket
Error Message: An error occurred (NoSuchBucket) when calling the PutBucketWebsite operation: The specified bucket does not exist
Incorrect method:
aws s3 website my-k21-buck --index-document index.html --error-document error.html
Explanation:
The index document is set to “index.html” and the error document is set to “error.html” by this command, which also enables hosting for static websites in the designated S3 bucket. If these files are missing from the bucket, trying to access the S3 bucket with an invalid bucket name or URL will result in a “NoSuchBucket” error.
Solution:
aws s3 website my-k21-bucket --index-document index.html --error-document error.html
Make sure that you have spelled the S3 bucket name correctly. If the bucket exists, ensure that the AWS CLI is using the correct region by specifying the –region parameter with the appropriate region name.
3. NoSuchKey/ Key does not exist
Error Message: fatal error: An error occurred (404) when calling the HeadObject operation: Key “hello.txt” does not exist
Incorrect method:
aws s3 cp s3://my-k21-bucket/hello.txt .
Explanation:
The above AWS CLI command is trying to copy the object “hello.txt” from S3 bucket ” my-k21-bucket” to local directory. However, if the object key (file name) provided in the command does not exist in the bucket, this command will result in a “NoSuchKey” error.
Solution:
To resolve this error, check the key or object name that you are trying to access and ensure that it exists in the specified S3 bucket. If the key or object exists, ensure that you have specified the correct S3 bucket name and key name in the AWS CLI command.
4. InvalidArgument
Error Message: aws: error: argument –grant-full-control: expected one argument.
Incorrect method:
aws s3api put-object-acl --bucket my-k21-bucket --key my-file.txt --acl invalid-acl-value --grant-full-control
Explanation:
In this example, we are trying to set an ACL and grant full control to an object (my-file.txt) in an S3 bucket (my-bucket-name) using the aws s3api put-object-acl command. However, we have specified an invalid ACL value (invalid-acl-value) which is not a valid value for the –acl parameter.
Solution:
aws s3api put-object-acl --bucket my-k21-bucket --key my-file.txt --grant-full-control id=0123456789
This command will upload a file to an S3 bucket and grant full control over the object to the AWS account with the specified ID (0123456789). Please note that this will not allow public read and write access to the object. If you also want to grant public read and write access, you can use a canned ACL like public-read-write or public-read instead of the grant-full-control parameter.
5. BucketAlreadyExists
Error message: make_bucket failed: s3://my-k21-bucket An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.
Explanation:
If you encounter this error, it means that the bucket you are trying to create already exists in AWS. To resolve the issue, you can either use the existing bucket, or you can choose a different name for your new bucket.
Solution:
If you want to check the list of your existing buckets:
aws s3 ls
If you want to create a new bucket with a different name:
aws s3api create-bucket --bucket my-new-k21-bucket --region my-region
My-region should be replaced with the AWS region where you wish to build your bucket, and my-new-k21-bucket should be replaced with a name that is specific to your bucket. Remember that bucket names must be universally distinct across all AWS accounts, so you might need to think of a more original name for your bucket.
6. NoSuchBucketPolicy
Error message: An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist
Incorrect method:
aws s3api get-bucket-policy --bucket my-k21-bucket
Explanation:
If you are trying to retrieve the bucket policy of an S3 bucket using the “get-bucket-policy” API and the specified bucket does not have a policy configured, you may encounter a “NoSuchBucketPolicy” error.
Solution:
To resolve this error, you can either create a new bucket policy for the bucket or remove the command that is trying to retrieve the bucket policy. If you want to create a new bucket policy for the bucket, you can use the following AWS CLI command:
vi policy.json
Paste below sample policy in the editor:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowAllActions", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": "s3:*", "Resource": [ "arn:aws:s3:::my-k21-bucket", "arn:aws:s3:::my-k21-bucket/*" ] } ] }
Type “:wq” (without quotations) and hit enter.
Now attach the policy:
aws s3api put-bucket-policy --bucket my-k21-bucket --policy file://policy.json
The correct way to configure a bucket policy for the S3 bucket “my-k21-bucket” is to use the “put-bucket-policy” API with the bucket name and the policy document file path. The correct AWS CLI command is shown above. Make sure the IAM user or role has the necessary permissions to put a bucket policy on the S3 bucket and public access is allowed to the bucket.
7. SlowDown
Error message: An error occurred (SlowDown) when calling the S3 operation: Please reduce your request rate
Incorrect method:
aws s3 ls s3://my-bucket –recursive
Explanation:
This command lists all the objects in the “my-bucket” S3 bucket, including all subdirectories, which can cause a high rate of requests to the bucket and potentially trigger a “SlowDown” error.
Solution:
aws s3 ls s3://my-bucket --recursive --no-human-readable --page-size 800
You can change the command to use the –page-size option, which restricts the number of objects returned per page, and the –no-human-readable option, which eliminates displaying the size of the objects displayed above, to get rid of the “SlowDown” issue. The “SlowDown” problem may be avoided by limiting the number of objects listed per page to 800 and lowering the number of queries made to the bucket.
8. AuthFailure
Error message: An error occurred (AuthFailure) when calling the S3 operation: AWS was not able to validate the provided access credentials
The config profile (non-existent-profile) could not be found
False approach:
aws s3 ls s3://my-bucket
Explanation:
This command lists the contents of the “my-bucket” S3 bucket, but if the credentials used to access the bucket are incorrect or have expired, or if the IAM user or role associated with the credentials does not have the necessary permissions to access the S3 bucket, it will result in an “AuthFailure” error.
Solution:
To resolve this error, you can verify that the AWS CLI credentials are correct and have not expired and ensure that the IAM user or role associated with the credentials has the necessary permissions to access the S3 bucket. You can also use the aws configure command to update the credentials and region used by the AWS CLI
aws s3 ls s3://my-bucket --profile my-profile
This command uses a special set of information called a “profile” to access the storage space. This profile has the correct login details and permissions needed to look at the files in the storage space. By using the correct profile, you can make sure that the command works properly and that you can see the files in the storage space without any login errors.
Frequently Asked Questions:
What are common AWS S3 errors encountered when using AWS CLI?
- AccessDenied
- NoSuchBucket
- NoSuchKey /Key does not exist
- InvalidArgument
- BucketAlreadyExists
- NoSuchBucketPolicy
- SlowDown
- AuthFailure
Which AWS CLI command is used to authenticate to aws?
The aws configure command is the quickest method for setting up your AWS CLI installation in ordinary use. The Amazon CLI asks you for the following information when you enter this command:
- Access key ID
- Secret Access Key
- AWS Region
- Output Format
Why does my AWS CLI S3 transfer fail with a 'Connection Timeout' error, and how can I fix it?
The occurrence of a 'Connection Timeout' error could result from problems with the network connection between your local device and the AWS S3 service. Typically, this error suggests that the request sent from the AWS CLI to the S3 service failed to get a response within the pre-set time limit, which is usually around 30 seconds. To fix this issue, try the following troubleshooting steps:
- Check your network connectivity
- Check the AWS S3 service status
- Increase the timeout value
- Try a different AWS region
- Check the size of the file being transferred
Related Links/References:
- AWS Certified Solutions Architect Associate SAA-C03 Exam details
- Amazon S3 Bucket: Overview, Errors & Resolutions
- How to Configure & Install AWS CLI?
- AWS Certified DevOps Engineer Professional DOP-C02
- How to Create a free tier Account in AWS?
Next Task For You
Begin your journey towards an AWS Cloud by joining our FREE Informative Class on Amazon Cloud Free Class by clicking on the below image.
Leave a Reply