The Globe and Mail Case Study

The Globe and Mail is Canada’s most read newspaper with a national weekly digital readership of 4.7 million. In print for 170 years, the newspaper delivers coverage of national, international, business, technology, arts, entertainment, and lifestyle news. A leader in the adoption of cutting-edge technology, The Globe has been working to offer its readers engaging content while expanding opportunities for advertisers. The newspaper recently launched an encrypted system to offer protection to whistleblowers and anonymous sources. It is also the first Canadian publisher to offer Moat Analytics to help its advertisers spend advertising dollars more effectively.

The Globe and Mail was planning to launch a new application that enables its growing online readership to access stories and breaking news from mobile devices. And to increase reader engagement, it wanted to serve up targeted articles based on each reader’s individual interests.

The Globe researched off-the-shelf personalized recommendation products, but none delivered the sophisticated customization that the newspaper wanted. “We wanted to offer a powerful tool that not only offers up content based on what people are currently reading, but uses complicated algorithms to factor in data such as what they’ve read in the past, where they live, the type of mobile device they’re using, and articles viewed by people with similar interests,” says Mike O’Neill, director of data science at The Globe and Mail. “The off-the-shelf tools we evaluated didn’t come close to offering the advanced level of personalization we were seeking.”

The Globe team considered building a custom system on premises, but concluded that hosting its article recommendation engine in the cloud would be faster and would provide greater flexibility for testing different algorithms. “Moving from the prototype stage to the production stage in the cloud has been a lot quicker than [it would have been] had we provisioned our own hardware,” says O’Neill. “In addition, we had an ambitious launch timeline, and the cloud made it possible to meet it.”

To help it develop a personalized recommendation system, The Globe and Mail tapped Amazon Web Services (AWS) and ClearScale, a San Francisco–based systems integration, application development, and managed services firm. “We had already been prototyping a number of recommendation algorithms on AWS, and it had been a great experience, so we decided to take the next step of using AWS for our official platform,” explains O’Neill.

An AWS Premier Consulting Partner, ClearScale helped The Globe implement its recommendation system quickly and cost-effectively. The firm partnered with The Globe to build a recommendation engine that can work at scale so that recommendations for the next set of articles are ready by the time the reader has loaded the current article. ClearScale quickly accomplished that goal, building a robust, customized personalized recommendation engine that can be expanded to The Globe’s web and mobile platforms without any changes—and completing its work under tight timelines. “The fact that AWS offers an end-to-end solution made the development process quick and seamless,” says ClearScale CEO Pavel Pragin. “The Globe was able to get the advanced customization capabilities it needed without the expense of piecing together disparate software.”

The Globe’s personalized recommendation system uses several AWS services. Amazon Kinesis is used to collect massive amounts of information including user clicks, article engagement, and article metadata. Amazon DynamoDB serves as the fast and flexible database in which to store this data. And Amazon Elastic MapReduce (Amazon EMR) processes the data, distributing the computational work across a cluster of virtual servers running on AWS.

Working with ClearScale, The Globe adopted a DevOps approach for infrastructure automation. ClearScale created templates for deploying each component of the infrastructure, which makes it more flexible, scalable, and redundant. Management of The Globe's production, staging, and development environments on AWS was made easier with AWS CloudFormation, which gives developers and systems administrators an easy way to create and manage a collection of AWS resources, and AWS OpsWorks, a configuration management service that helps users configure and operate applications using Chef automation. With these tools in place, conditions such as CPU and memory utilization are automatically monitored, eliminating the need to manually add servers.

The newspaper also uses Amazon Simple Storage Service (Amazon S3) for secure, durable, highly scalable object storage, Amazon Elastic Cloud Compute (Amazon EC2) for resizable compute capacity, Amazon Simple Queue Service (Amazon SQS) for queuing messages, and Amazon CloudWatch to monitor the performance of the system. “For the solution to work, everything had to be completely automated,” says Pragin. “Moreover, all the components had to be 100 percent scalable to accommodate spikes in user demand and to allow the newspaper to grow.”

The personal recommendation engine has positioned The Globe and Mail to deliver relevant content to its audiences at a time of evolving online choices for news and information. “I was impressed with ClearScale’s ability to deliver such a complex yet robust, custom system within our tight timeless,” O’Neill says. “They are extremely capable—we chose the right vendor.”

By building its personalized recommendation system on AWS, The Globe and Mail was able to get the service to market in just three months—less than half the time it would have taken had the newspaper built an on-premises solution. “I don’t think The Globe would have been able to do it that fast without AWS,” says Pragin. “Had they built the infrastructure on-premises, it would have taken a long time and been expensive, and the solution wouldn’t have had the scalability capabilities the company needed.”

Building its solution on AWS, The Globe has also obtained a flexible platform for testing, allowing the company to improve its mobile app over time. “Working with ClearScale and AWS, we developed a solution that allows us to easily test different algorithms and variations of our app,” says O’Neill. “As a result, we can better understand what is and isn’t working, so we can continually improve the performance of our app and quickly measure the lift we’re getting.”

Perhaps most importantly, the newspaper is dramatically increasing reader engagement. Initial results show that parts of the mobile app that promote a personalized selection of articles based on the recommendation engine are seeing 25 percent greater click-through than a selection of the current most popular articles that would otherwise be the top performers in terms of click-through. The Globe and Mail anticipates this lift in engagement may grow even higher as it completes repeated rounds of champion-challenger testing to continually improve its algorithms.

With a robust personalized recommendation system in place, The Globe and Mail now offers its online audience a wealth of relevant content, and keeps readers engaged longer without forcing them to spend a lot of time searching for stories of interest. “One of our key goals has been to increase engagement,” says O’Neill. “And as we work to improve the reader experience, AWS and ClearScale are helping us get there both quickly and cost-effectively.”