Navigating the Cloud: Key Performance Indicators for Success

The worldwide public cloud market is poised for substantial growth in upcoming years, with “end-user spending projected to reach $679 billion in 2024.[1]” . This surge is fueled by enterprises’ ongoing migration of existing workloads, the development of new cloud-native applications, and the emergence of innovative use cases like generative AI.

At AWS, our Cloud Economics team has developed the Cloud Value Framework (CVF) to highlight the business value of cloud computing across various pillars, such as cost savings, staff productivity, operational resilience, business agility, and sustainability. We collaborate closely with organizations to map their transformation journeys and quantify outcomes across these CVF pillars, providing comprehensive five-year Total Cost of Ownership projections. However, tracking progress and ensuring ROI as business conditions evolve is vital.

This is where key performance indicators (KPIs) play a crucial role. They offer a standardized methodology based on inputs to evaluate an organization’s cloud maturity across technology and business dimensions. This blog post dives deeper into KPIs for technology, and a follow-up blog will focus on business dimensions.

Understanding KPIs

KPIs are valuable tools for assessing an organization’s performance in relation to specific goals and objectives. While unit metrics offer granular measurements of specific activities and are valuable for measuring the impact of cost-saving initiatives, KPIs provide a comprehensive perspective, enabling organizations to concentrate on strategic priorities and offering a broader view of overall impact. Given their significance and the input required from multiple stakeholders, implementing KPIs can be challenging. Nevertheless, their importance justifies the effort required. By leveraging these indicators, organizations can make well-informed decisions about the need for continuous improvement or corrective action to ensure their strategies are on track.

Choosing the Right KPIs

Selecting the appropriate KPIs demands careful consideration. Here are some guidelines:

• Keep it concise: Use no more than five KPIs to avoid overwhelming stakeholders.
• Use SMART criteria: KPIs should be Specific, Measurable, Attainable, Relevant, and Time-Bound (SMART).
• Make it actionable: Focus on input parameters and their correlation to output to measure the impact of specific actions.

When determining which KPIs to track, the focus should be on achieving internal alignment, streamlining data collection, and establishing governance. KPIs should be reviewed by relevant executives and stakeholders, data collection should be standardized, and processes for reporting and corrective actions should be established. It’s also important to consider the ongoing retrospective mechanism, which evaluates whether the chosen KPIs are still valid over time.

KPI Examples by Value Pillars

Selecting KPIs to measure business value should focus on areas that directly measure impact on customer satisfaction, operational efficiencies, revenue growth, etc. While no one-size-fits-all approach exists, some common themes have emerged across organizations. Here are KPI examples across value pillars:

Cost Savings

• Cost savings trend analysis measures whether cost in the cloud is trending in the right direction, with incremental costs in the cloud increasing at a lower rate than the decrease in on-premises costs.
• Cloud efficiency ratio is cloud spend divided by revenue. It measures the cost efficiency of cloud spend versus how revenue changes through time.
• Average utilization rate of infrastructure measures how effectively AWS compute, databases, and data warehouse services are utilized. As organization maturity increases, the utilization rate of resources should trend upward.
• Cost of training or fine-tuning generative AI allows enterprises to measure the cost associated with generative AI, such as compute resources, data acquisition, and human annotation effort, as a foundation to calculate ROI.

Staff Productivity

• Lead time to deploy new account and infrastructure components measures the time to provision new accounts with common infrastructure components that meet organizational finance, security, and architectural guidelines. Additional measurements can include the time to deploy compute, database, storage, etc.
• The number of virtual machines managed per administrator measures operational efficiency improvement due to the reduced scope of responsibility for administrators. For the KPI to be successful, enterprises need to implement additional measures for organizational change management.
• Defect density (to measure the number of defects per unit of code) measures the number of defects found per unit of code, which should show defect density decreasing as various developer productivity tools are utilized. Units of code can be specific functions, lines of code, etc.
• Cost avoidance due to generative AI measures the cost savings achieved by automating tasks or processes previously performed manually or through traditional methods such as chat support, summarization, general content generation, repetitive software code and unit tests, etc.

Operational Resilience

• Total amount of downtime (or average downtime per incident) measures the total amount of unplanned downtime or unplanned downtime per incident. Employing automated detection and observability mechanisms helps measure how quickly a response is initiated after the incident.
• Number of security incidents due to misconfiguration measures the number of security incidents due to resource misconfiguration or security posture vulnerability.
• Mean Time to Resolution measures whether the average repair time decreases as organizations develop faster recovery and response time using cloud-native mechanisms.
• Average time to detect and respond measures whether incident response time decreases due to proactive notification and response mechanisms.
• SLA breach penalty (for organizations with defined SLA as part of customer contracts or regulatory obligations) measures whether the total SLA breach penalty divided by the total number of customers decreases over time due to various security and resiliency mechanisms.

Business Agility

• Release frequency and the average number of new updates/features per release measure the number of new releases in a specific period and the number of new features introduced per release.
• Number of code reviews per release and the average time spent per code review measure the reduction in manual code reviews per release as well as staff efficiency improvement per code review meeting.
• Number of applications using microservices architecture measures the number of applications moving from monolithic to microservices architecture to remove interdependencies.
• Innovation rate using generative AI tracks the number of new ideas, concepts, or solutions generated by generative AI that can lead to business opportunities or competitive advantages.

Sustainability

• Green computing initiatives measure carbon footprint reduction due to savings from a reduction in on-premises IT operations, a combination of renewable energy usage and infrastructure efficiency of the cloud, as well as the use of managed services.
• Waste reduction metrics measure the percentage of on-premises infrastructure waste recycled or disposed of in an environmentally friendly manner rather than in landfills.

Governance and Conclusion

Governance of KPIs demands executive alignment, consistent data collection processes, and corrective actions when negative trends arise. By focusing on trend analysis and periodic refinement, organizations can accurately gauge the pace of cloud adoption, optimize efficiency, and highlight tangible benefits to stakeholders.

Selecting the appropriate KPIs to gauge the business value of a cloud migration requires thoughtful consideration. While optimizing the infrastructure footprint and realizing cost savings in cloud is imperative, tracking the progress of the journey and its impact on internal stakeholders and external customers can also provide a competitive edge. Common KPIs emerge centered on five pillars of value regardless of industry sector. However, each organization’s unique landscape and priorities shape its best-suited KPIs. Securing stakeholder alignment and systematizing consistent data gathering is vital for success.

As conditions shift, existing KPIs need periodic reevaluation and refinement. Trend analysis grants more meaningful insights than judging solitary data points. Ultimately, the right KPIs exhibit whether your digital transformation drives strategic objectives and delivers differentiated business value over time.

KPIs provide unique measurements of your digital transformation’s health and allow you to build on success because what gets managed, measured, and reported gets done.

[1] Gartner® Press Release, Gartner Forecasts Worldwide Public Cloud End-User Spending to Reach $679 Billion in 2024, November 13, 2023, https://www.gartner.com/en/newsroom/press-releases/11-13-2023-gartner-forecasts-worldwide-public-cloud-end-user-spending-to-reach-679-billion-in-20240.  GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.