Snap Optimizes Cost Savings While Storing Over 1.5 Trillion Photos and Videos on Amazon S3 Glacier Instant Retrieval

In 2016, Snap migrated its data to AWS. “We chose to migrate to AWS because of its global reach, excellent performance, and competitive pricing that, in turn, gave us the ability to reinvest in our business,” says Vijay Manoharan, manager of the media delivery platform team at Snap.

Snap’s needs accelerated in 2016 after the launch of Snapchat Memories, a feature that automatically archives media and resurfaces it over time. “Snapchat Memories is our predominant use case for storing media for long periods,” says Manoharan. Snapchatters might view this content for a few days and then not view it again for months or years, so the company wanted to optimize its storage on AWS for further cost savings.

Snap had been storing saved media on Amazon S3 Standard-Infrequent Access (S3 Standard-IA), a storage class for data that is infrequently accessed (once every 1–2 months) but requires rapid access when needed. With the launch of Amazon S3 Glacier Instant Retrieval in November 2021, the company realized that it could save even more on costs with virtually no impact on performance. The Snap team even influenced the development of this archive storage class by providing feedback and collaborating with the Amazon S3 team as the storage class was being designed. To determine if Amazon S3 Glacier Instant Retrieval delivered a lower total cost than Amazon S3 Standard-IA, Snap began by analyzing the access patterns of its data. This analysis showed that using Amazon S3 Glacier Instant Retrieval would reduce costs because the storage class is ideal for data that needs immediate access but is only accessed once per quarter. So, Snap began migrating to the storage class in March 2022 using Amazon S3 Lifecycle policies. By June 2022, Snap had migrated all existing content and was storing all new content in Amazon S3 Glacier Instant Retrieval.

Snap migrated more than 2 exabytes of data—roughly equivalent to 1.5 trillion media files—seamlessly to Amazon S3 Glacier Instant Retrieval from Amazon S3 Standard-IA. “The fact that no customer noticed this major migration to Amazon S3 Glacier Instant Retrieval was a big win for us,” says Manoharan. “It was a seamless experience for Snapchatters, and we had no production issues during the entire migration.” As a result of the migration, the company saved tens of millions of dollars on storage. Snap has configured Amazon S3 in 20 AWS Regions around the world so that customers anywhere can retrieve data in milliseconds. The AWS Global Infrastructure is the most secure, extensive, and reliable Global Cloud Infrastructure for a business’s applications. The global reach of AWS lets Snap store media closer to the place where Snapchatters are creating it for optimal performance. Snap is also able to deliver content efficiently using Amazon CloudFront, a content delivery network service built for high performance, security, and availability. “We’ve been able to off-load all of the regionalization work and costs to AWS so that we can focus on developing new features,” says Manoharan. As a result, Snapchat continues to meet its quarterly cost-optimization goals.

Migrating Snap’s content to Amazon S3 has also improved operations and visibility. Using Amazon S3 Storage Lens, a feature that delivers organization-wide visibility into object storage usage, the company has better insight into what it’s storing so that it can make more informed, data-driven decisions. Snap also migrated to AWS to scale its infrastructure to support its growth: the amount of content that it stores has grown by 5–10 percent each year. Meanwhile, Snap transitioned other parts of its infrastructure from its previous monolithic architecture to one based on microservices to host many of the services that powered its app. To accomplish this, it turned to Amazon Elastic Kubernetes Service (Amazon EKS), a managed container service to run and scale Kubernetes applications in the cloud or on premises. “We worked extensively with the AWS team to migrate some of our features and components to microservices on AWS,” says Manoharan. Each microservice can be deployed in multiple Regions, simplifying the management of its infrastructure. As a result, Snap saw a 20–30 percent reduction in download latency in certain Regions for refreshing feeds, downloading media, and doing near-real-time communications.

In 2017, Snap migrated one of the app’s most central features—Snapchat Stories—to Amazon DynamoDB, a fully managed, serverless, NoSQL database designed to run high-performance applications at virtually any scale. Using Amazon DynamoDB, the company experienced greater than 99.99 percent availability and can better manage the metadata associated with customers’ photos and videos. The company estimates that it has added 200 million daily active users since 2016 and has dramatically improved its ability to grow and innovate on AWS.

Snap plans to continue looking for opportunities to achieve further cost savings while focusing on innovation. “The AWS team provided us with tremendous support,” says Manoharan. “That commitment has really helped us prioritize our business needs.”

On AWS, Snap is ready to handle more growth and roll out innovative features in a way that’s both cost efficient and delivers a great user experience. “By gaining new insights on AWS,” Manoharan says, “we can strike the right balance between further reducing costs and maintaining performance.”