Expedia Group Scales Database-as-a-Service Platform Using AWS Service Catalog

Expedia Group operates travel sites in more than 70 countries, offering flights from over 500 airlines and bookings for over three million properties. To support its operations, Expedia Group runs over 9,000 applications across more than 400 AWS accounts. Expedia Group’s various teams also use about two dozen different database technologies, and not all application teams have the expertise to manage databases effectively. To simplify and drive efficiency, the company has begun strategically standardizing the tools its teams use, and it has chosen to expand the Cerebro DBaaS platform—which had been created internally at Vrbo, an Expedia Group brand—across the whole organization as a standard solution for database provisioning and management.

Expanding Cerebro across Expedia Group meant redesigning the platform for scalability while keeping it manageable for a small team to support. Expedia Group used AWS Service Catalog, a service for organizations to create and manage catalogs of IT services that are approved for use on AWS, to scale out the platform. The company selected the service after conducting a proof of concept that demonstrated that AWS Service Catalog would work well for its use case. Using AWS Service Catalog, Expedia Group engineers can centrally manage deployed IT services and achieve consistent governance across accounts. “Using Cerebro built on AWS Service Catalog, we provide an efficient way for our developers to provision and manage databases,” says Hao Nguyen, senior director of data engineering at Expedia Group. “We can onboard new technology, scale up and down based on demand, and provide governance in terms of security and cost optimization for databases across Expedia Group, not just one division.”

Using AWS Service Catalog, the Cerebro DBaaS platform was scaled up from managing database provisioning for three AWS accounts to managing over 400 AWS accounts. The platform manages the deployment and maintenance of multiple database engines, including NoSQL, graph databases, open-source databases, and AWS-managed services, including Amazon Aurora, a relational database management system built for the cloud with full MySQL and PostgreSQL compatibility. “We support multiple database technologies based on different needs, and we want our users to have a seamless deployment experience regardless of the database type,” says Nguyen. “We need a scalable and flexible platform to do that.” The DBaaS platform helps Expedia Group enforce consistent governance, security controls, and best practices across teams by setting parameters automatically when developers provision a new database. Additionally, the company can improve cost optimization because Cerebro provides a clear overview of what resources are being used for which applications.

Cerebro makes it simple for Expedia Group developers to provision database resources quickly for themselves. The platform uses AWS CloudFormation, a service that lets users model, provision, and manage AWS and third-party resources by treating infrastructure as code. Developers use the AWS Service Catalog console to view the products that they can deploy, input basic information such as database type and tags, and then launch the database. Cerebro takes care of the underlying network configurations and sets up monitoring and security parameters automatically by pushing CloudFormation templates into the new accounts. Previously, developers had to wait a few days after submitting a ticket to the database team, but now developers can deploy a database themselves in only a few minutes. Because the process is so simple, developers can spend more time focusing on their work rather than on database configurations. “Our teams are constantly innovating to provide new features and products to meet customer needs,” says Nguyen. “Cerebro helps provide a more efficient, seamless experience to our developers.”

Cerebro also uses AWS Systems Manager, a complete management solution for hybrid cloud environments, to detect performance anomalies and perform self-healing, remediation, and other database management tasks. Using Systems Manager, Expedia Group’s database support engineers can connect to all the instances that have been provisioned using Cerebro from a single account to do maintenance and patching. The company also reduced work hours spent manually resolving database issues by automating its response to alerts. Using a variety of AWS and third-party solutions—including Amazon Simple Notification Service (Amazon SNS), a fully managed messaging service for application-to-application and application-to-person communication—the Cerebro First Responder system runs scripts to resolve database alerts. The most common alerts can be resolved automatically, with the system escalating support to a person only if necessary. Using automated troubleshooting helps Expedia Group—a global company with travel customers in any time zone—stay available without fatiguing its support team.

Expedia Group expects the number of production databases that are provisioned using Cerebro to increase from around 5,000 nodes to over 20,000 nodes in the next 2 years while keeping the support headcount to a minimum. The company will continue to fine-tune the platform, and the team has recently added SQL database engines to the list of technologies that it supports.

“Teams that have onboarded our DBaaS platform don’t need to worry about data infrastructure anymore,” says Nguyen. “Using AWS, we can scale out the platform as needed. It’s simple to manage, so now our teams can focus resources on innovating and developing services to power travel across the globe.”