How do I calculate the total network usage for my EC2 instances and determine which instance used the most bandwidth within a date range?

Short description

Use the Amazon CloudWatch NetworkIn and NetworkOut metrics to determine which Amazon EC2 instance is causing high network usage. You can aggregate the data points from these metrics to calculate the network usage for your EC2 instance.

Note: The CloudWatch NetworkOut metric is a different metric from the DataTransfer-Out-Bytes metric that's used in AWS Cost Explorer reports. The CloudWatch NetworkOut metric represents the amount of outbound network traffic from an instance, regardless of where it went.

Prerequisites for using CloudWatch metrics:

• Set up an AWS Command Line Interface (AWS CLI) environment. For more information, see What is the AWS Command Line Interface? If you receive errors when running AWS CLI commands, make sure that you're using the most recent version of the AWS CLI.
• Turn on the cloudwatch:GetMetricStatistics permission for the AWS Identity and Access Management (IAM) user or role that you use to run the AWS CLI. See Actions, resources, and condition keys for Amazon CloudWatch for more information.
• Have a Bash shell to run the script.

Note: The following resolution works only for instances that you can see in the Amazon EC2 dashboard. The following Bash shell script doesn't work for terminated instances. For terminated instances, add the instance IDs to the script in a space separated format.

Example space separated format:

ADD_INSTANCES="i-xxxx i-yyyy i-zzzz"

Resolution

The following Bash shell script identifies the network usage of all EC2 instances in an AWS Region.

1.    Copy the following content to create a script:

#!/bin/bash
if [ $# -ne 4 ]; then
echo "Usage: $0 <REGION> <NetworkIn|NetworkOut> <START_TIMESTAMP> <END_TIMESTAMP>"
echo -e "\tNote: Do not change the order of parameters."
echo -e "\n\tExample: $0 ap-south-1 NetworkOut 2020-06-01T00:00:00.000Z 2020-06-30T23:59:59.000Z"
exit 1
fi
REGION="$1"
METRIC="$2"
START_TIME="$3"
END_TIME="$4"
ADD_INSTANCES=""
INSTANCES="${ADD_INSTANCES} $(aws ec2 describe-instances --region ${REGION} --query Reservations[*].Instances[*].InstanceId --output text)" || { echo "Failed to run aws ec2 describe-instances commandline, exiting..."; exit 1; }
[ "${INSTANCES}x" == "x" ] && { echo "There are no instances found from the given region ${REGION}, exiting..."; exit 1; }
for _instance_id in ${INSTANCES}; do
unset _value
_value="$(aws cloudwatch get-metric-statistics --metric-name ${METRIC} --start-time ${START_TIME} --end-time ${END_TIME} --period 86400 --namespace AWS/EC2 --statistics Sum --dimensions Name=InstanceId,Value=${_instance_id} --region ${REGION} --output text)"
[ "${_value}x" == "x" ] && { echo "Something went wrong while calculating the network usage of ${_instance_id}"; continue; }
echo "${_instance_id}: $(echo "${_value}" | awk '{ sum += $2 } END {printf ("%f\n", sum/1024/1024/1024)}';) GiB";
done
echo -e "\nNote: If you think the values are inaccurate, please verify the input and modify if needed."

2.    Save the script with any name. For example, network_usage.sh.

network_usage.sh

3.    Run the following command to turn on permission for the script to run:

$ sudo chmod a+x network_usage.sh

4.    Use the following syntax to run the script:

$ bash network_usage.sh ap-south-1 NetworkOut 2020-06-01T00:00:00.000Z 2020-06-30T23:59:59.000Z

• Replace ap-south-1 with the Region where your instances are located. For more information, see Amazon EC2 Available Regions.
• Specify either NetworkIn or NetworkOut to calculate the traffic flow direction.
• Replace 2020-06-01T00:00:00.000Z and 2020-06-30T23:59:59.000Z with the start and end timestamps in the range that you want to calculate the network usage for.

The following example shows a sample output from the preceding command:

i-025a820f5ee9f1490: 0.000099 GiB
i-0b56df07325d9c0fa: 0.008475 GiB
i-09ad0a93b60a233b1: 0.000000 GiB
i-0ba35b570c7003ffe: 0.000000 GiB
i-012cc851406584b5c: 0.000000 GiB
i-0ef3921993739d772: 219.884000 GiB
i-01f193931c4101cf3: 0.000000 GiB
i-05ca8ccc8aa7caf76: 0.009235 GiB
i-06b165356b5769581: 0.000000 GiB
i-06c4d1e1cb4e9139d: 0.012911 GiB
i-070b6508dd4ab5040: 0.000000 GiB
i-0f29e3ba7f156dfee: 0.000000 GiB

Related information

List the available CloudWatch metrics for your instances

GNU Bash (from the GNU website)