Getting started with Graviton

Start by watching these two demos, "AWS re:Invent 2024 - AWS Graviton: The best price performance for your AWS workloads" and "re:Invent 2021 – Deep dive into AWS Graviton3 and Amazon EC2 C7g instances" which will give you an overview of the Graviton-based instances and some insights on how to run applications depending on their operating system, languages, and runtimes.  

Keep learning about silicon innovation to better understand Amazon’s commitment to pushing the envelope with custom silicon. Spend some time reading relevant sections of the AWS Graviton Technical Guide in GitHub repository, which will act as a useful reference throughout your adoption.

Top Tip: Task 2 will require you to build the application/workload list of dependencies. If you need information from other teams, it might be useful to ask them for this information before you start the first task, so that you could have their answers by the time you reach Task 2.

To start the move, the first thing you need to do is inventory your current software stack so you can identify the path to equivalent software versions that support Graviton. At this stage, it can be useful to think in terms of software you download (e.g. open source packages, container images, libraries), software you build, and software you procure/license (e.g. monitoring or security agents).

Areas to review:

• Operating system and version (the more recent the better).

• If your workload is container based, check container images you consume for Arm64 support. Keep in mind many container images now support multiple architectures, which simplifies consumption of those images in a mixed-architecture environment. You can read ECR multiarch support announcement for more details.

• All the libraries, frameworks, and runtimes used by the application and its components.

• The tools used to build, deploy, and test your application (e.g. compilers, test suites, CI/CD pipelines, provisioning tools, and scripts).

• All the tools and/or agents used to deploy and manage the application in production (e.g. monitoring tools or security agents).

• The Porting Advisor for Graviton is an open-source command line tool that analyzes source code and generates a report highlighting missing and outdated libraries and code constructs that may require modification along with recommendations for alternatives. It accelerates your transition to AWS Graviton-based instances by reducing the iterative process of identifying and resolving source code and library dependencies.

There are language-specific sections in the AWS Graviton Technical Guide in GitHub with useful pointers to get the best performance from Graviton processors as well as guidance for Operating Systems, Container environments, and various open-source software.

For each component of your software stack, check for arm64/Graviton support. A large portion of this can be done using existing configuration scripts, as your scripts run and install packages you will get messages for any missing components, some may build from source automatically while others will cause the script to fail. Pay attention to software versions as in general the more current your software is the easier the transition, and the more likely you’ll achieve the full performance entitlement from Graviton processors. If you do need to perform upgrades prior to adopting Graviton then it is best to do that using an existing x86 environment to minimize the number of changed variables. We have seen examples where upgrading OS version on x86 was far more involved and time consuming than transitioning to Graviton after the upgrade

When locating software be aware that some tools, including GCC, refer to the architecture as AArch64, others including the Linux Kernel, call it arm64. When checking packages across various repositories, you’ll find those different naming conventions.

If you find any software without support for the Arm64, please let AWS know by reaching out on the re:Post community support forum.

The main ways to check and places to look include:

• Package repositories of your chosen Linux distribution(s). Arm64 support within Linux distributions is largely complete: for example, Debian, which has the largest package repository, has over 98% of its packages built for the arm64 architecture.

• Container image registry. Amazon ECR now offers public repositories that you can search for arm64 images. DockerHub allows you to search for a specific architecture (arm64 enabled images).  

Specific to containers, you may find an amd64 (x86-64) container image you currently use transitioned to a multi-architecture container image when adding Arm64 support. This means you may not find an explicit arm64 container, so be sure to check for both as projects may chose to vend discrete images for x86-64 and arm64 while other projects chose to vend a multi-arch image supporting both architectures.

• On GitHub, you can check for arm64 versions in the release section. However, some projects don’t use the release section, or only release source archives, so you may need to visit the main project webpage and check the download section. You can also search the GitHub project for “arm64” or “AArch64” to see whether the project has any arm64 code contributions or issues. Even if a project does not currently produce builds for arm64, in many cases an Arm64 version of those packages will be available through Linux distributions or additional package repositories. You can search for packages using a package search tool such as pkgs.org.

• In the download section or platform support matrix of your software vendors, look for references to Arm64, AArch64, or Graviton. Software vendor documentation will often list ‘platform requirements’ which include supported operating system versions and architectures.

Categories of software with potential issues:

• Packages or applications sourced from an ISV may not yet be available for Graviton. AWS is working with many software partners to offer technical guidance as they add support for Graviton, but some are still missing or in the process of adding support. A non-exhaustive list of some ISV software can be found in here.

• The Python community produces modules built using low level languages (e.g. C/C++) that need to be compiled for the Arm64 architecture. You may use modules that are not currently available as pre-built binaries from the Python Package Index. AWS is actively working with open-source communities to ensure the most popular modules are available. In the meantime we provide specific instructions to resolve the build-time dependencies for missing packages in the Python section of the Graviton Getting Started Guide.

To transition and test your application, you will first need a Graviton environment, so depending on your execution environment, you’ll have to:

• Obtain or create an Arm64 AMI to boot your Graviton instance(s) from. Depending on how you manage your AMIs, you can either start directly from an existing reference AMI for Arm64, or you can build your Golden AMI with your specific dependencies from one of the reference images. See this full list of supported operating systems with AMI links.

• If you operate a container-based environment, you’ll need to build or extend an existing cluster with support for Graviton-based instances. Both Amazon ECS and EKS support adding Graviton-based instances to an existing x86-based cluster. For ECS, you can add Graviton-based instances to your ECS cluster, launching them with either the AWS ECS-optimized AMI for arm64 or your own AMI after you’ve installed the ECS agent. For EKS, you will need to create a node-group with Graviton-based instances launched with the EKS optimized AMI for arm64.

You can support Graviton and x86 instances in the same Auto Scaling Group using the launch template override.

• Complete the installation of your software stack based on the inventory created in Task 2.

Note: if you are not building your application or component parts of your overall application stack, then you may skip this step.

Now that you have an environment available, you can build your application stack. For applications built using interpreted or JIT languages, including Java, PHP, or Node.js, they should run as-is or with only minor modifications. The repository contains language-specific sections with recommendations, such as Java, Python, C/C++, Golang, PHP, R, Node.js, Rust, or .NET.

Note: if there is no language specific section, it is because there is no specific guidance beyond using a suitably current version of the language relevant to Graviton. .NET-core is a great way to benefit from Graviton-based instances, this blog post covers .NET 8 performance.

Applications using compiled languages, including C, C++, or Go, need to be compiled for the Arm64 architecture. Most modern builds (e.g. using Make) will work when run natively on Graviton-based instances, however, you’ll find language-specific compiler recommendations in these repositories: C/C++, Go, and Rust.

Just like an operating system, container images are architecture-specific. You will need to build Arm64 container image(s). To make the transition easier, we recommend building multi-arch container image(s) that can run automatically on either x86-64 or Arm64. Check out the container section of the repository for more details and this detailed overview of multi-architecture container image support for Amazon ECR, which is considered a best practice for establishing and maintaining a multi architecture environment.

If you believe you are observing architecture-specific issues, please check the Arm Architecture Reference Manual Armv8 or reach out to us on the re:Post community support forum.

You will also need to review any functional and unit test suite(s) to ensure you can test the new build artifacts with the same test coverage you have already for x86 artifacts.  

Now that you have your application stack on Graviton, you should run your test suite to ensure all regular unit and functional tests pass. Resolve any test failures in the application(s) or test suites until you are satisfied everything is working as expected. Most errors should be related to the modifications and updated software versions you have installed during the transition. If you suspect architecture-specific issue(s) please have a look at the C/C++ section in Task 4, which documents them and gives advice on how to solve them. If there are still details that seem unclear or you are not observing the expected application performance, please reach out to your AWS account team, or to the AWS support for assistance.

When upgrading software versions, first test them using an existing x86 environment to minimize the number of variables changed at a time. If issues occur, then resolve them using the current x86 environment before continuing with the new Graviton environment.

With your fully functional application, it is time to establish a performance baseline on Graviton. In many cases, Graviton will provide performance and/or capacity improvements over x86-based instances.

One of the major differences between AWS Graviton instances and other Amazon EC2 instances is their vCPU-to-physical-core mappings. Every vCPU on a Graviton processor maps to a physical core, and there is no Simultaneous Multi-Threading (SMT). Consequently, Graviton provides better linear performance scalability in most cases. When comparing to existing x86 instances, we recommend fully loading both instance types to determine the maximum sustainable load before the latency or error rate exceeds acceptable bounds. For horizontally-scalable, multi-threaded workloads that are CPU bound, you may find that the Graviton instances are able to sustain a significantly higher transaction rate before unacceptable latencies or error rates occur.

During the transition to Graviton, if you are using Amazon EC2 Auto Scaling, you may be able to increase the threshold values for the CloudWatch alarms that invoke the scaling process. This may reduce the number of EC2 instances now needed to serve a given level of demand.

Important: This repository has sections dedicated to Optimization and a Performance Runbook for you to follow during this stage.

If, after reading the documentation in this repository and following the recommendations, you do not observe expected performance, then please reach out to your AWS account team, or send an email to ec2-arm-dev-feedback@amazon.com with details so we can assist you with your performance observations.

Now that you have a tested and performant application, it’s time to update your infrastructure as code to add support for Graviton-based instances. This typically includes updating instance types, AMI IDs, ASG constructs to support multi-architecture (see Amazon EC2 ASG support for multiple Launch Templates), and finally deploying or redeploying your infrastructure.

Once your infrastructure is ready to support Graviton-based instances, you can start a Canary or Blue-Green deployment to re-direct a portion of application traffic to the Graviton-based instances. Ideally initial tests will run in a development environment to load test with production traffic patterns. Monitor the application closely to ensure expected behavior. Once your application is running as expected on Graviton you can define and execute your transition strategy and begin to enjoy the benefits of increased price-performance.