How do I troubleshoot slow or inconsistent speeds when downloading or uploading to Amazon S3?

Resolution

Check the following to identify and mitigate what might be contributing to slow or inconsistent speeds when downloading or uploading to Amazon S3:

• The location of the client making requests
• The client internet service provider (ISP)
• The client network
• The client resources
• The request rate to Amazon S3
• The Amazon S3 server access logs to determine the Turn-Around Time

The location of the client making requests

Use Amazon S3 buckets that are as geographically close to your client as possible. Buckets are globally accessible, but they reside in a specific AWS Region. The geographical distance between the request and the bucket contributes to the time that it takes for a response to be received.

You can test the impact of geographical distance between the client and the S3 bucket. For example, you can launch an Amazon Elastic Compute Cloud (Amazon EC2) instance in the same AWS Region as the bucket. Then, launch another instance in a different Region. Test an upload and download of the same file using both instances, and then compare the throughput between the two Regions.

To decrease the distance between the client and the S3 bucket, move your data into a bucket in another Region that's closer to the client. You can configure cross-Region replication so that data in the source bucket is replicated into the destination bucket in the new Region. As another option, consider migrating the client closer to the S3 bucket.

The client internet service provider (ISP)

Review your network path for any issues that affect how your connection traverses the internet. These issues can include packet loss, high number of hops, or other ISP-related issues.

Tools such as mtr and traceroute can help provide clues for potential packet loss. They can also provide the latency of the packets that are sent to the remote host. Additionally, these tools can help identify if there's a network hop causing higher latency.

For example, the following Linux traceroute command uses TCP port 80 to test connectivity to the us-west-2 (Oregon) Regional endpoint:

sudo traceroute -P TCP -p 80 s3.us-west-2.amazonaws.com

On Windows operating systems, you can use a similar tool called tracert.

Note: Many network devices don't respond to ICMP. For a test that resembles requests to Amazon S3, be sure to run a TCP traceroute or mtr to the Regional endpoint of the bucket.

If the internet route from the client to Amazon S3 isn't optimal, consider using Amazon S3 Transfer Acceleration, which leverages Amazon CloudFront's edge locations. Review the Amazon S3 Transfer Acceleration Speed Comparison tool to see if Transfer Acceleration can improve performance for your use case.

Note: If you turn on Transfer Acceleration, additional data transfer charges might apply. Be sure to review Amazon S3 pricing for data transfers.

The client network

Confirm that your client network is healthy by reviewing internal packet inspections, antivirus scanning, or network access management. Additionally, review how your client or application handles DNS resolution and caching.

To benefit from Amazon S3's distributed nature and availability, avoid caching the DNS resolution for long periods of time.

The client resources

There might be latency introduced in your application or how your host that's making the requests is handling the requests sent and responses received. As a best practice, confirm that there's no resource contention within the host that might be contributing to the overall latency. For example, the resource contention within the host might be with the CPU, memory, or network bandwidth.

You can check the resource usage during data transfer on most client systems using tools such as Resource Monitor or the top command. The tools available depend on your operating system.

The client's storage device or system might also be a source of latency. Read or write operations to the client's storage device that experience elevated latency can impact the performance of the download or upload to Amazon S3. You must troubleshoot the IOPS of the storage device from the client side. You can analyze Amazon S3 performance with the Amazon CloudWatch metric FirstByteLatency.

Note: Amazon S3 CloudWatch request metrics are billed at the same rate as custom metrics.

FirstByteLatency shows how long it takes for Amazon S3 to process the request from the client and then begin sending the response to the client. The CloudWatch metric provides a bucket-level perspective of performance. You can narrow down your investigation using Amazon S3 server access logs. For more information, see the section Amazon S3 server access logs to determine the Turn-Around Time.

The request rate to Amazon S3

By default, S3 buckets can support thousands of requests per second per prefix. If a client is getting HTTP 5xx error responses from Amazon S3, this can indicate that the supported request rate per prefix is exceeded. To troubleshoot 5xx errors, see How do I troubleshoot an HTTP 500 or 503 error from Amazon S3?

The Amazon S3 server access logs to determine the Turn-Around Time

You can turn on Amazon S3 server access logging to review Turn-Around Time metrics. Turn-Around Time is the time between when the last byte of the request is received and when the first byte of the response is sent. This is a Time To First Byte (TTFB) metric. You can use Turn-Around Time to determine the amount of time the operation is taking from an Amazon S3 perspective. Then, you can evaluate other latencies outside of Amazon S3 that might be contributing to an overall slow download or upload.

If you see latency that's higher than usual, it's a best practice to retry the request instead of waiting for the request to complete. For more information on this guidance and other performance recommendations, see Performance guidelines for Amazon S3.